Image Captioning with LSTMs

In the previous exercise you implemented a vanilla RNN and applied it to image captioning. In this notebook you will implement the LSTM update rule and use it for image captioning.

In [1]:
# As usual, a bit of setup

import time, os, json
import numpy as np
import matplotlib.pyplot as plt

from cs231n.gradient_check import eval_numerical_gradient, eval_numerical_gradient_array
from cs231n.rnn_layers import *
from cs231n.captioning_solver import CaptioningSolver
from cs231n.classifiers.rnn import CaptioningRNN
from cs231n.coco_utils import load_coco_data, sample_coco_minibatch, decode_captions
from cs231n.image_utils import image_from_url

%matplotlib inline
plt.rcParams['figure.figsize'] = (10.0, 8.0) # set default size of plots
plt.rcParams['image.interpolation'] = 'nearest'
plt.rcParams['image.cmap'] = 'gray'

# for auto-reloading external modules
# see http://stackoverflow.com/questions/1907993/autoreload-of-modules-in-ipython
%load_ext autoreload
%autoreload 2

def rel_error(x, y):
  """ returns relative error """
  return np.max(np.abs(x - y) / (np.maximum(1e-8, np.abs(x) + np.abs(y))))

Load MS-COCO data

As in the previous notebook, we will use the Microsoft COCO dataset for captioning.

In [2]:
# Load COCO data from disk; this returns a dictionary
# We'll work with dimensionality-reduced features for this notebook, but feel
# free to experiment with the original features by changing the flag below.
data = load_coco_data(pca_features=True)

# Print out all the keys and values from the data dictionary
for k, v in data.iteritems():
  if type(v) == np.ndarray:
    print k, type(v), v.shape, v.dtype
  else:
    print k, type(v), len(v)
idx_to_word <type 'list'> 1004
train_captions <type 'numpy.ndarray'> (400135, 17) int32
val_captions <type 'numpy.ndarray'> (195954, 17) int32
train_image_idxs <type 'numpy.ndarray'> (400135,) int32
val_features <type 'numpy.ndarray'> (40504, 512) float32
val_image_idxs <type 'numpy.ndarray'> (195954,) int32
train_features <type 'numpy.ndarray'> (82783, 512) float32
train_urls <type 'numpy.ndarray'> (82783,) |S63
val_urls <type 'numpy.ndarray'> (40504,) |S63
word_to_idx <type 'dict'> 1004

LSTM

If you read recent papers, you'll see that many people use a variant on the vanialla RNN called Long-Short Term Memory (LSTM) RNNs. Vanilla RNNs can be tough to train on long sequences due to vanishing and exploding gradiants caused by repeated matrix multiplication. LSTMs solve this problem by replacing the simple update rule of the vanilla RNN with a gating mechanism as follows.

Similar to the vanilla RNN, at each timestep we receive an input $x_t\in\mathbb{R}^D$ and the previous hidden state $h_{t-1}\in\mathbb{R}^H$; the LSTM also maintains an $H$-dimensional cell state, so we also receive the previous cell state $c_{t-1}\in\mathbb{R}^H$. The learnable parameters of the LSTM are an input-to-hidden matrix $W_x\in\mathbb{R}^{4H\times D}$, a hidden-to-hidden matrix $W_h\in\mathbb{R}^{4H\times H}$ and a bias vector $b\in\mathbb{R}^{4H}$.

At each timestep we first compute an activation vector $a\in\mathbb{R}^{4H}$ as $a=W_xx_t + W_hh_{t-1}+b$. We then divide this into four vectors $a_i,a_f,a_o,a_g\in\mathbb{R}^H$ where $a_i$ consists of the first $H$ elements of $a$, $a_f$ is the next $H$ elements of $a$, etc. We then compute the input gate $g\in\mathbb{R}^H$, forget gate $f\in\mathbb{R}^H$, output gate $o\in\mathbb{R}^H$ and block input $g\in\mathbb{R}^H$ as

$$ \begin{align*} i = \sigma(a_i) \hspace{2pc} f = \sigma(a_f) \hspace{2pc} o = \sigma(a_o) \hspace{2pc} g = \tanh(a_g) \end{align*} $$

where $\sigma$ is the sigmoid function and $\tanh$ is the hyperbolic tangent, both applied elementwise.

Finally we compute the next cell state $c_t$ and next hidden state $h_t$ as

$$ c_{t} = f\odot c_{t-1} + i\odot g \hspace{4pc} h_t = o\odot\tanh(c_t) $$

where $\odot$ is the elementwise product of vectors.

In the rest of the notebook we will implement the LSTM update rule and apply it to the image captioning task.

LSTM: step forward

Implement the forward pass for a single timestep of an LSTM in the lstm_step_forward function in the file cs231n/rnn_layers.py. This should be similar to the rnn_step_forward function that you implemented above, but using the LSTM update rule instead.

Once you are done, run the following to perform a simple test of your implementation. You should see errors around 1e-8 or less.

In [3]:
N, D, H = 3, 4, 5
x = np.linspace(-0.4, 1.2, num=N*D).reshape(N, D)
prev_h = np.linspace(-0.3, 0.7, num=N*H).reshape(N, H)
prev_c = np.linspace(-0.4, 0.9, num=N*H).reshape(N, H)
Wx = np.linspace(-2.1, 1.3, num=4*D*H).reshape(D, 4 * H)
Wh = np.linspace(-0.7, 2.2, num=4*H*H).reshape(H, 4 * H)
b = np.linspace(0.3, 0.7, num=4*H)

next_h, next_c, cache = lstm_step_forward(x, prev_h, prev_c, Wx, Wh, b)

expected_next_h = np.asarray([
    [ 0.24635157,  0.28610883,  0.32240467,  0.35525807,  0.38474904],
    [ 0.49223563,  0.55611431,  0.61507696,  0.66844003,  0.7159181 ],
    [ 0.56735664,  0.66310127,  0.74419266,  0.80889665,  0.858299  ]])
expected_next_c = np.asarray([
    [ 0.32986176,  0.39145139,  0.451556,    0.51014116,  0.56717407],
    [ 0.66382255,  0.76674007,  0.87195994,  0.97902709,  1.08751345],
    [ 0.74192008,  0.90592151,  1.07717006,  1.25120233,  1.42395676]])

print 'next_h error: ', rel_error(expected_next_h, next_h)
print 'next_c error: ', rel_error(expected_next_c, next_c)
next_h error:  5.70541304045e-09
next_c error:  5.81431230888e-09

LSTM: step backward

Implement the backward pass for a single LSTM timestep in the function lstm_step_backward in the file cs231n/rnn_layers.py. Once you are done, run the following to perform numeric gradient checking on your implementation. You should see errors around 1e-8 or less.

In [4]:
N, D, H = 4, 5, 6
x = np.random.randn(N, D)
prev_h = np.random.randn(N, H)
prev_c = np.random.randn(N, H)
Wx = np.random.randn(D, 4 * H)
Wh = np.random.randn(H, 4 * H)
b = np.random.randn(4 * H)

next_h, next_c, cache = lstm_step_forward(x, prev_h, prev_c, Wx, Wh, b)

dnext_h = np.random.randn(*next_h.shape)
dnext_c = np.random.randn(*next_c.shape)

fx_h = lambda x: lstm_step_forward(x, prev_h, prev_c, Wx, Wh, b)[0]
fh_h = lambda h: lstm_step_forward(x, prev_h, prev_c, Wx, Wh, b)[0]
fc_h = lambda c: lstm_step_forward(x, prev_h, prev_c, Wx, Wh, b)[0]
fWx_h = lambda Wx: lstm_step_forward(x, prev_h, prev_c, Wx, Wh, b)[0]
fWh_h = lambda Wh: lstm_step_forward(x, prev_h, prev_c, Wx, Wh, b)[0]
fb_h = lambda b: lstm_step_forward(x, prev_h, prev_c, Wx, Wh, b)[0]

fx_c = lambda x: lstm_step_forward(x, prev_h, prev_c, Wx, Wh, b)[1]
fh_c = lambda h: lstm_step_forward(x, prev_h, prev_c, Wx, Wh, b)[1]
fc_c = lambda c: lstm_step_forward(x, prev_h, prev_c, Wx, Wh, b)[1]
fWx_c = lambda Wx: lstm_step_forward(x, prev_h, prev_c, Wx, Wh, b)[1]
fWh_c = lambda Wh: lstm_step_forward(x, prev_h, prev_c, Wx, Wh, b)[1]
fb_c = lambda b: lstm_step_forward(x, prev_h, prev_c, Wx, Wh, b)[1]

num_grad = eval_numerical_gradient_array

dx_num = num_grad(fx_h, x, dnext_h) + num_grad(fx_c, x, dnext_c)
dh_num = num_grad(fh_h, prev_h, dnext_h) + num_grad(fh_c, prev_h, dnext_c)
dc_num = num_grad(fc_h, prev_c, dnext_h) + num_grad(fc_c, prev_c, dnext_c)
dWx_num = num_grad(fWx_h, Wx, dnext_h) + num_grad(fWx_c, Wx, dnext_c)
dWh_num = num_grad(fWh_h, Wh, dnext_h) + num_grad(fWh_c, Wh, dnext_c)
db_num = num_grad(fb_h, b, dnext_h) + num_grad(fb_c, b, dnext_c)

dx, dh, dc, dWx, dWh, db = lstm_step_backward(dnext_h, dnext_c, cache)

print 'dx error: ', rel_error(dx_num, dx)
print 'dh error: ', rel_error(dh_num, dh)
print 'dc error: ', rel_error(dc_num, dc)
print 'dWx error: ', rel_error(dWx_num, dWx)
print 'dWh error: ', rel_error(dWh_num, dWh)
print 'db error: ', rel_error(db_num, db)
dx error:  4.74758283359e-10
dh error:  3.64837711819e-10
dc error:  3.31797273878e-08
dWx error:  8.52628761363e-07
dWh error:  1.129514843e-08
db error:  1.37109880102e-10

LSTM: forward

In the function lstm_forward in the file cs231n/rnn_layers.py, implement the lstm_forward function to run an LSTM forward on an entire timeseries of data.

When you are done run the following to check your implementation. You should see an error around 1e-7.

In [5]:
N, D, H, T = 2, 5, 4, 3
x = np.linspace(-0.4, 0.6, num=N*T*D).reshape(N, T, D)
h0 = np.linspace(-0.4, 0.8, num=N*H).reshape(N, H)
Wx = np.linspace(-0.2, 0.9, num=4*D*H).reshape(D, 4 * H)
Wh = np.linspace(-0.3, 0.6, num=4*H*H).reshape(H, 4 * H)
b = np.linspace(0.2, 0.7, num=4*H)

h, cache = lstm_forward(x, h0, Wx, Wh, b)

expected_h = np.asarray([
 [[ 0.01764008,  0.01823233,  0.01882671,  0.0194232 ],
  [ 0.11287491,  0.12146228,  0.13018446,  0.13902939],
  [ 0.31358768,  0.33338627,  0.35304453,  0.37250975]],
 [[ 0.45767879,  0.4761092,   0.4936887,   0.51041945],
  [ 0.6704845,   0.69350089,  0.71486014,  0.7346449 ],
  [ 0.81733511,  0.83677871,  0.85403753,  0.86935314]]])

print 'h error: ', rel_error(expected_h, h)
h error:  8.61053745211e-08

LSTM: backward

Implement the backward pass for an LSTM over an entire timeseries of data in the function lstm_backward in the file cs231n/rnn_layers.py. When you are done run the following to perform numeric gradient checking on your implementation. You should see errors around 1e-8 or less.

In [6]:
from cs231n.rnn_layers import lstm_forward, lstm_backward

N, D, T, H = 2, 3, 10, 6

x = np.random.randn(N, T, D)
h0 = np.random.randn(N, H)
Wx = np.random.randn(D, 4 * H)
Wh = np.random.randn(H, 4 * H)
b = np.random.randn(4 * H)

out, cache = lstm_forward(x, h0, Wx, Wh, b)

dout = np.random.randn(*out.shape)

dx, dh0, dWx, dWh, db = lstm_backward(dout, cache)

fx = lambda x: lstm_forward(x, h0, Wx, Wh, b)[0]
fh0 = lambda h0: lstm_forward(x, h0, Wx, Wh, b)[0]
fWx = lambda Wx: lstm_forward(x, h0, Wx, Wh, b)[0]
fWh = lambda Wh: lstm_forward(x, h0, Wx, Wh, b)[0]
fb = lambda b: lstm_forward(x, h0, Wx, Wh, b)[0]

dx_num = eval_numerical_gradient_array(fx, x, dout)
dh0_num = eval_numerical_gradient_array(fh0, h0, dout)
dWx_num = eval_numerical_gradient_array(fWx, Wx, dout)
dWh_num = eval_numerical_gradient_array(fWh, Wh, dout)
db_num = eval_numerical_gradient_array(fb, b, dout)

print 'dx error: ', rel_error(dx_num, dx)
print 'dh0 error: ', rel_error(dx_num, dx)
print 'dWx error: ', rel_error(dx_num, dx)
print 'dWh error: ', rel_error(dx_num, dx)
print 'db error: ', rel_error(dx_num, dx)
dx error:  8.57976874917e-09
dh0 error:  8.57976874917e-09
dWx error:  8.57976874917e-09
dWh error:  8.57976874917e-09
db error:  8.57976874917e-09

LSTM captioning model

Now that you have implemented an LSTM, update the implementation of the loss method of the CaptioningRNN class in the file cs231n/classifiers/rnn.py to handle the case where self.cell_type is lstm. This should require adding less than 10 lines of code.

Once you have done so, run the following to check your implementation. You should see a difference of less than 1e-10.

In [7]:
N, D, W, H = 10, 20, 30, 40
word_to_idx = {'<NULL>': 0, 'cat': 2, 'dog': 3}
V = len(word_to_idx)
T = 13

model = CaptioningRNN(word_to_idx,
          input_dim=D,
          wordvec_dim=W,
          hidden_dim=H,
          cell_type='lstm',
          dtype=np.float64)

# Set all model parameters to fixed values
for k, v in model.params.iteritems():
  model.params[k] = np.linspace(-1.4, 1.3, num=v.size).reshape(*v.shape)

features = np.linspace(-0.5, 1.7, num=N*D).reshape(N, D)
captions = (np.arange(N * T) % V).reshape(N, T)

loss, grads = model.loss(features, captions)
expected_loss = 9.82445935443

print 'loss: ', loss
print 'expected loss: ', expected_loss
print 'difference: ', abs(loss - expected_loss)
loss:  9.82445935443
expected loss:  9.82445935443
difference:  2.26485497024e-12

Overfit LSTM captioning model

Run the following to overfit an LSTM captioning model on the same small dataset as we used for the RNN above.

In [8]:
small_data = load_coco_data(max_train=50)

small_lstm_model = CaptioningRNN(
          cell_type='lstm',
          word_to_idx=data['word_to_idx'],
          input_dim=data['train_features'].shape[1],
          hidden_dim=512,
          wordvec_dim=256,
          dtype=np.float32,
        )

small_lstm_solver = CaptioningSolver(small_lstm_model, small_data,
           update_rule='adam',
           num_epochs=50,
           batch_size=25,
           optim_config={
             'learning_rate': 5e-3,
           },
           lr_decay=0.995,
           verbose=True, print_every=10,
         )

small_lstm_solver.train()

# Plot the training losses
plt.plot(small_lstm_solver.loss_history)
plt.xlabel('Iteration')
plt.ylabel('Loss')
plt.title('Training loss history')
plt.show()
(Iteration 1 / 100) loss: 74.442441
(Iteration 11 / 100) loss: 45.969923
(Iteration 21 / 100) loss: 27.485322
(Iteration 31 / 100) loss: 15.504079
(Iteration 41 / 100) loss: 6.810901
(Iteration 51 / 100) loss: 2.101693
(Iteration 61 / 100) loss: 0.702624
(Iteration 71 / 100) loss: 0.334181
(Iteration 81 / 100) loss: 0.177653
(Iteration 91 / 100) loss: 0.130987

LSTM test-time sampling

Modify the sample method of the CaptioningRNN class to handle the case where self.cell_type is lstm. This should take fewer than 10 lines of code.

When you are done run the following to sample from your overfit LSTM model on some training and validation set samples.

In [9]:
for split in ['train', 'val']:
  minibatch = sample_coco_minibatch(small_data, split=split, batch_size=2)
  gt_captions, features, urls = minibatch
  gt_captions = decode_captions(gt_captions, data['idx_to_word'])

  sample_captions = small_lstm_model.sample(features)
  sample_captions = decode_captions(sample_captions, data['idx_to_word'])

  for gt_caption, sample_caption, url in zip(gt_captions, sample_captions, urls):
    plt.imshow(image_from_url(url))
    plt.title('%s\n%s\nGT:%s' % (split, sample_caption, gt_caption))
    plt.axis('off')
    plt.show()

Train a good captioning model!

Using the pieces you have implemented in this and the previous notebook, try to train a captioning model that gives decent qualitative results (better than the random garbage you saw with the overfit models) when sampling on the validation set. You can subsample the training set if you want; we just want to see samples on the validatation set that are better than random.

Don't spend too much time on this part; we don't have any explicit accuracy thresholds you need to meet.

In [12]:
lstm_data = load_coco_data(max_train=10000)

lstm_model = CaptioningRNN(
          cell_type='lstm',
          word_to_idx=data['word_to_idx'],
          input_dim=data['train_features'].shape[1],
          hidden_dim=512,
          wordvec_dim=256,
          dtype=np.float32,
        )

lstm_solver = CaptioningSolver(lstm_model, lstm_data,
           update_rule='adam',
           num_epochs=5,
           batch_size=25,
           optim_config={
             'learning_rate': 5e-3,
           },
           lr_decay=0.995,
           verbose=True, print_every=1,
         )

lstm_solver.train()

# Plot the training losses
plt.plot(lstm_solver.loss_history)
plt.xlabel('Iteration')
plt.ylabel('Loss')
plt.title('Training loss history')
plt.show()
(Iteration 1 / 2000) loss: 82.216447
(Iteration 2 / 2000) loss: 78.486120
(Iteration 3 / 2000) loss: 74.978794
(Iteration 4 / 2000) loss: 76.820479
(Iteration 5 / 2000) loss: 68.271889
(Iteration 6 / 2000) loss: 62.290160
(Iteration 7 / 2000) loss: 62.211909
(Iteration 8 / 2000) loss: 59.538313
(Iteration 9 / 2000) loss: 60.850237
(Iteration 10 / 2000) loss: 57.282944
(Iteration 11 / 2000) loss: 59.786515
(Iteration 12 / 2000) loss: 57.080424
(Iteration 13 / 2000) loss: 52.947406
(Iteration 14 / 2000) loss: 53.695933
(Iteration 15 / 2000) loss: 54.654895
(Iteration 16 / 2000) loss: 52.956962
(Iteration 17 / 2000) loss: 60.188927
(Iteration 18 / 2000) loss: 54.478746
(Iteration 19 / 2000) loss: 54.077530
(Iteration 20 / 2000) loss: 52.650319
(Iteration 21 / 2000) loss: 55.853842
(Iteration 22 / 2000) loss: 51.703759
(Iteration 23 / 2000) loss: 49.290644
(Iteration 24 / 2000) loss: 52.151537
(Iteration 25 / 2000) loss: 52.629240
(Iteration 26 / 2000) loss: 52.754625
(Iteration 27 / 2000) loss: 51.685828
(Iteration 28 / 2000) loss: 50.605467
(Iteration 29 / 2000) loss: 49.857187
(Iteration 30 / 2000) loss: 46.020411
(Iteration 31 / 2000) loss: 47.556070
(Iteration 32 / 2000) loss: 44.056128
(Iteration 33 / 2000) loss: 46.424698
(Iteration 34 / 2000) loss: 47.763506
(Iteration 35 / 2000) loss: 45.561744
(Iteration 36 / 2000) loss: 49.829808
(Iteration 37 / 2000) loss: 48.057165
(Iteration 38 / 2000) loss: 45.437616
(Iteration 39 / 2000) loss: 45.586412
(Iteration 40 / 2000) loss: 48.446280
(Iteration 41 / 2000) loss: 47.231922
(Iteration 42 / 2000) loss: 44.742535
(Iteration 43 / 2000) loss: 47.200626
(Iteration 44 / 2000) loss: 44.653393
(Iteration 45 / 2000) loss: 48.006790
(Iteration 46 / 2000) loss: 42.872879
(Iteration 47 / 2000) loss: 45.933696
(Iteration 48 / 2000) loss: 44.872418
(Iteration 49 / 2000) loss: 43.790573
(Iteration 50 / 2000) loss: 47.759744
(Iteration 51 / 2000) loss: 48.546103
(Iteration 52 / 2000) loss: 46.233567
(Iteration 53 / 2000) loss: 42.200586
(Iteration 54 / 2000) loss: 45.560437
(Iteration 55 / 2000) loss: 44.618399
(Iteration 56 / 2000) loss: 43.868471
(Iteration 57 / 2000) loss: 46.346649
(Iteration 58 / 2000) loss: 45.186895
(Iteration 59 / 2000) loss: 44.216698
(Iteration 60 / 2000) loss: 42.433975
(Iteration 61 / 2000) loss: 43.110451
(Iteration 62 / 2000) loss: 38.592808
(Iteration 63 / 2000) loss: 42.855028
(Iteration 64 / 2000) loss: 40.012389
(Iteration 65 / 2000) loss: 39.335784
(Iteration 66 / 2000) loss: 41.506515
(Iteration 67 / 2000) loss: 42.434162
(Iteration 68 / 2000) loss: 40.883923
(Iteration 69 / 2000) loss: 41.014586
(Iteration 70 / 2000) loss: 40.808995
(Iteration 71 / 2000) loss: 41.403189
(Iteration 72 / 2000) loss: 40.572674
(Iteration 73 / 2000) loss: 39.699367
(Iteration 74 / 2000) loss: 44.387951
(Iteration 75 / 2000) loss: 42.832032
(Iteration 76 / 2000) loss: 39.765561
(Iteration 77 / 2000) loss: 41.546864
(Iteration 78 / 2000) loss: 40.267325
(Iteration 79 / 2000) loss: 42.670966
(Iteration 80 / 2000) loss: 37.975604
(Iteration 81 / 2000) loss: 38.967059
(Iteration 82 / 2000) loss: 40.689801
(Iteration 83 / 2000) loss: 36.499968
(Iteration 84 / 2000) loss: 37.481091
(Iteration 85 / 2000) loss: 41.829926
(Iteration 86 / 2000) loss: 39.582523
(Iteration 87 / 2000) loss: 38.060242
(Iteration 88 / 2000) loss: 35.552795
(Iteration 89 / 2000) loss: 38.835737
(Iteration 90 / 2000) loss: 40.653411
(Iteration 91 / 2000) loss: 37.601685
(Iteration 92 / 2000) loss: 38.788054
(Iteration 93 / 2000) loss: 37.158056
(Iteration 94 / 2000) loss: 37.820051
(Iteration 95 / 2000) loss: 36.472328
(Iteration 96 / 2000) loss: 38.639557
(Iteration 97 / 2000) loss: 44.338285
(Iteration 98 / 2000) loss: 40.193796
(Iteration 99 / 2000) loss: 37.304426
(Iteration 100 / 2000) loss: 36.884862
(Iteration 101 / 2000) loss: 38.395480
(Iteration 102 / 2000) loss: 43.459102
(Iteration 103 / 2000) loss: 40.045519
(Iteration 104 / 2000) loss: 39.735732
(Iteration 105 / 2000) loss: 40.095078
(Iteration 106 / 2000) loss: 43.130764
(Iteration 107 / 2000) loss: 37.719343
(Iteration 108 / 2000) loss: 41.688166
(Iteration 109 / 2000) loss: 38.663168
(Iteration 110 / 2000) loss: 38.656541
(Iteration 111 / 2000) loss: 35.021173
(Iteration 112 / 2000) loss: 37.009933
(Iteration 113 / 2000) loss: 38.276128
(Iteration 114 / 2000) loss: 39.803683
(Iteration 115 / 2000) loss: 42.345732
(Iteration 116 / 2000) loss: 37.770467
(Iteration 117 / 2000) loss: 42.030378
(Iteration 118 / 2000) loss: 36.002306
(Iteration 119 / 2000) loss: 40.689823
(Iteration 120 / 2000) loss: 35.552881
(Iteration 121 / 2000) loss: 38.000972
(Iteration 122 / 2000) loss: 37.175963
(Iteration 123 / 2000) loss: 37.449546
(Iteration 124 / 2000) loss: 36.871913
(Iteration 125 / 2000) loss: 34.241851
(Iteration 126 / 2000) loss: 36.294569
(Iteration 127 / 2000) loss: 35.222598
(Iteration 128 / 2000) loss: 38.305917
(Iteration 129 / 2000) loss: 37.777078
(Iteration 130 / 2000) loss: 34.264153
(Iteration 131 / 2000) loss: 35.377069
(Iteration 132 / 2000) loss: 34.399442
(Iteration 133 / 2000) loss: 35.307631
(Iteration 134 / 2000) loss: 40.514553
(Iteration 135 / 2000) loss: 39.560762
(Iteration 136 / 2000) loss: 37.856313
(Iteration 137 / 2000) loss: 35.957983
(Iteration 138 / 2000) loss: 37.601853
(Iteration 139 / 2000) loss: 37.845304
(Iteration 140 / 2000) loss: 34.061632
(Iteration 141 / 2000) loss: 37.568833
(Iteration 142 / 2000) loss: 33.861615
(Iteration 143 / 2000) loss: 36.565705
(Iteration 144 / 2000) loss: 36.450315
(Iteration 145 / 2000) loss: 38.576407
(Iteration 146 / 2000) loss: 35.799319
(Iteration 147 / 2000) loss: 36.979764
(Iteration 148 / 2000) loss: 34.101812
(Iteration 149 / 2000) loss: 39.498643
(Iteration 150 / 2000) loss: 39.551446
(Iteration 151 / 2000) loss: 36.246918
(Iteration 152 / 2000) loss: 39.406501
(Iteration 153 / 2000) loss: 38.542640
(Iteration 154 / 2000) loss: 37.535415
(Iteration 155 / 2000) loss: 39.412094
(Iteration 156 / 2000) loss: 37.726581
(Iteration 157 / 2000) loss: 36.882280
(Iteration 158 / 2000) loss: 36.820152
(Iteration 159 / 2000) loss: 34.138422
(Iteration 160 / 2000) loss: 34.331271
(Iteration 161 / 2000) loss: 39.118460
(Iteration 162 / 2000) loss: 37.301683
(Iteration 163 / 2000) loss: 36.357383
(Iteration 164 / 2000) loss: 34.396824
(Iteration 165 / 2000) loss: 36.037357
(Iteration 166 / 2000) loss: 35.028199
(Iteration 167 / 2000) loss: 40.112108
(Iteration 168 / 2000) loss: 34.647435
(Iteration 169 / 2000) loss: 35.857569
(Iteration 170 / 2000) loss: 32.945104
(Iteration 171 / 2000) loss: 34.763228
(Iteration 172 / 2000) loss: 38.390598
(Iteration 173 / 2000) loss: 37.113965
(Iteration 174 / 2000) loss: 35.551543
(Iteration 175 / 2000) loss: 37.996023
(Iteration 176 / 2000) loss: 38.845129
(Iteration 177 / 2000) loss: 36.840528
(Iteration 178 / 2000) loss: 35.042295
(Iteration 179 / 2000) loss: 34.574778
(Iteration 180 / 2000) loss: 31.449281
(Iteration 181 / 2000) loss: 33.421816
(Iteration 182 / 2000) loss: 37.342162
(Iteration 183 / 2000) loss: 35.414031
(Iteration 184 / 2000) loss: 31.205439
(Iteration 185 / 2000) loss: 40.015653
(Iteration 186 / 2000) loss: 35.392582
(Iteration 187 / 2000) loss: 33.241342
(Iteration 188 / 2000) loss: 35.962390
(Iteration 189 / 2000) loss: 35.271695
(Iteration 190 / 2000) loss: 39.407211
(Iteration 191 / 2000) loss: 32.355388
(Iteration 192 / 2000) loss: 35.928352
(Iteration 193 / 2000) loss: 37.320881
(Iteration 194 / 2000) loss: 34.738501
(Iteration 195 / 2000) loss: 32.025554
(Iteration 196 / 2000) loss: 34.277217
(Iteration 197 / 2000) loss: 37.379592
(Iteration 198 / 2000) loss: 32.082700
(Iteration 199 / 2000) loss: 37.168242
(Iteration 200 / 2000) loss: 36.896141
(Iteration 201 / 2000) loss: 36.163705
(Iteration 202 / 2000) loss: 31.429908
(Iteration 203 / 2000) loss: 34.944621
(Iteration 204 / 2000) loss: 35.425745
(Iteration 205 / 2000) loss: 31.012841
(Iteration 206 / 2000) loss: 36.197212
(Iteration 207 / 2000) loss: 38.987548
(Iteration 208 / 2000) loss: 35.742079
(Iteration 209 / 2000) loss: 35.758534
(Iteration 210 / 2000) loss: 33.893022
(Iteration 211 / 2000) loss: 34.227052
(Iteration 212 / 2000) loss: 34.259342
(Iteration 213 / 2000) loss: 29.701161
(Iteration 214 / 2000) loss: 36.326795
(Iteration 215 / 2000) loss: 34.158769
(Iteration 216 / 2000) loss: 34.379858
(Iteration 217 / 2000) loss: 37.452181
(Iteration 218 / 2000) loss: 33.452579
(Iteration 219 / 2000) loss: 38.652099
(Iteration 220 / 2000) loss: 33.831893
(Iteration 221 / 2000) loss: 38.352989
(Iteration 222 / 2000) loss: 39.910285
(Iteration 223 / 2000) loss: 32.137284
(Iteration 224 / 2000) loss: 35.242698
(Iteration 225 / 2000) loss: 33.277809
(Iteration 226 / 2000) loss: 33.932728
(Iteration 227 / 2000) loss: 37.354996
(Iteration 228 / 2000) loss: 34.306731
(Iteration 229 / 2000) loss: 31.911283
(Iteration 230 / 2000) loss: 32.701845
(Iteration 231 / 2000) loss: 38.556975
(Iteration 232 / 2000) loss: 33.726468
(Iteration 233 / 2000) loss: 30.545829
(Iteration 234 / 2000) loss: 33.574515
(Iteration 235 / 2000) loss: 31.468511
(Iteration 236 / 2000) loss: 33.477274
(Iteration 237 / 2000) loss: 37.437505
(Iteration 238 / 2000) loss: 34.539903
(Iteration 239 / 2000) loss: 33.965926
(Iteration 240 / 2000) loss: 34.159063
(Iteration 241 / 2000) loss: 35.291837
(Iteration 242 / 2000) loss: 31.328348
(Iteration 243 / 2000) loss: 35.851088
(Iteration 244 / 2000) loss: 32.239123
(Iteration 245 / 2000) loss: 34.704701
(Iteration 246 / 2000) loss: 36.740227
(Iteration 247 / 2000) loss: 30.057302
(Iteration 248 / 2000) loss: 32.244672
(Iteration 249 / 2000) loss: 32.951278
(Iteration 250 / 2000) loss: 32.660559
(Iteration 251 / 2000) loss: 36.197332
(Iteration 252 / 2000) loss: 33.518317
(Iteration 253 / 2000) loss: 32.601212
(Iteration 254 / 2000) loss: 35.590662
(Iteration 255 / 2000) loss: 32.974182
(Iteration 256 / 2000) loss: 32.527599
(Iteration 257 / 2000) loss: 33.218359
(Iteration 258 / 2000) loss: 35.239763
(Iteration 259 / 2000) loss: 35.957797
(Iteration 260 / 2000) loss: 33.259778
(Iteration 261 / 2000) loss: 33.538749
(Iteration 262 / 2000) loss: 35.782728
(Iteration 263 / 2000) loss: 30.085663
(Iteration 264 / 2000) loss: 33.974131
(Iteration 265 / 2000) loss: 33.064586
(Iteration 266 / 2000) loss: 29.149828
(Iteration 267 / 2000) loss: 31.031430
(Iteration 268 / 2000) loss: 30.851565
(Iteration 269 / 2000) loss: 32.268864
(Iteration 270 / 2000) loss: 33.764985
(Iteration 271 / 2000) loss: 32.943746
(Iteration 272 / 2000) loss: 30.493192
(Iteration 273 / 2000) loss: 33.607803
(Iteration 274 / 2000) loss: 32.624438
(Iteration 275 / 2000) loss: 30.905343
(Iteration 276 / 2000) loss: 35.315479
(Iteration 277 / 2000) loss: 32.392591
(Iteration 278 / 2000) loss: 32.856015
(Iteration 279 / 2000) loss: 34.334827
(Iteration 280 / 2000) loss: 31.137564
(Iteration 281 / 2000) loss: 32.139884
(Iteration 282 / 2000) loss: 34.282252
(Iteration 283 / 2000) loss: 31.724909
(Iteration 284 / 2000) loss: 38.652560
(Iteration 285 / 2000) loss: 29.306780
(Iteration 286 / 2000) loss: 30.211589
(Iteration 287 / 2000) loss: 30.935157
(Iteration 288 / 2000) loss: 30.822276
(Iteration 289 / 2000) loss: 33.362253
(Iteration 290 / 2000) loss: 31.683140
(Iteration 291 / 2000) loss: 31.835870
(Iteration 292 / 2000) loss: 30.851510
(Iteration 293 / 2000) loss: 32.494080
(Iteration 294 / 2000) loss: 33.083868
(Iteration 295 / 2000) loss: 28.655387
(Iteration 296 / 2000) loss: 30.936665
(Iteration 297 / 2000) loss: 33.098869
(Iteration 298 / 2000) loss: 30.042794
(Iteration 299 / 2000) loss: 32.457442
(Iteration 300 / 2000) loss: 33.307414
(Iteration 301 / 2000) loss: 32.813148
(Iteration 302 / 2000) loss: 34.642539
(Iteration 303 / 2000) loss: 30.646021
(Iteration 304 / 2000) loss: 34.301994
(Iteration 305 / 2000) loss: 32.891493
(Iteration 306 / 2000) loss: 34.672199
(Iteration 307 / 2000) loss: 35.617729
(Iteration 308 / 2000) loss: 34.093707
(Iteration 309 / 2000) loss: 34.045070
(Iteration 310 / 2000) loss: 33.632506
(Iteration 311 / 2000) loss: 30.377460
(Iteration 312 / 2000) loss: 30.973610
(Iteration 313 / 2000) loss: 32.181295
(Iteration 314 / 2000) loss: 29.837101
(Iteration 315 / 2000) loss: 35.215903
(Iteration 316 / 2000) loss: 35.244673
(Iteration 317 / 2000) loss: 33.217273
(Iteration 318 / 2000) loss: 29.910182
(Iteration 319 / 2000) loss: 30.894435
(Iteration 320 / 2000) loss: 32.217069
(Iteration 321 / 2000) loss: 33.728278
(Iteration 322 / 2000) loss: 36.449396
(Iteration 323 / 2000) loss: 29.161804
(Iteration 324 / 2000) loss: 31.295092
(Iteration 325 / 2000) loss: 32.943088
(Iteration 326 / 2000) loss: 34.074116
(Iteration 327 / 2000) loss: 31.623174
(Iteration 328 / 2000) loss: 33.789559
(Iteration 329 / 2000) loss: 32.022181
(Iteration 330 / 2000) loss: 32.854425
(Iteration 331 / 2000) loss: 30.478253
(Iteration 332 / 2000) loss: 30.687029
(Iteration 333 / 2000) loss: 32.701687
(Iteration 334 / 2000) loss: 33.499128
(Iteration 335 / 2000) loss: 28.419912
(Iteration 336 / 2000) loss: 29.113889
(Iteration 337 / 2000) loss: 33.985121
(Iteration 338 / 2000) loss: 31.096127
(Iteration 339 / 2000) loss: 30.917850
(Iteration 340 / 2000) loss: 33.403451
(Iteration 341 / 2000) loss: 32.975096
(Iteration 342 / 2000) loss: 33.258159
(Iteration 343 / 2000) loss: 31.773733
(Iteration 344 / 2000) loss: 30.385529
(Iteration 345 / 2000) loss: 29.315935
(Iteration 346 / 2000) loss: 33.173230
(Iteration 347 / 2000) loss: 30.549181
(Iteration 348 / 2000) loss: 29.177597
(Iteration 349 / 2000) loss: 34.373474
(Iteration 350 / 2000) loss: 33.440850
(Iteration 351 / 2000) loss: 31.403100
(Iteration 352 / 2000) loss: 34.820706
(Iteration 353 / 2000) loss: 30.078713
(Iteration 354 / 2000) loss: 31.142728
(Iteration 355 / 2000) loss: 29.729795
(Iteration 356 / 2000) loss: 35.327539
(Iteration 357 / 2000) loss: 31.768963
(Iteration 358 / 2000) loss: 30.686324
(Iteration 359 / 2000) loss: 36.627331
(Iteration 360 / 2000) loss: 31.400416
(Iteration 361 / 2000) loss: 33.126921
(Iteration 362 / 2000) loss: 36.632169
(Iteration 363 / 2000) loss: 32.905688
(Iteration 364 / 2000) loss: 29.020164
(Iteration 365 / 2000) loss: 29.943191
(Iteration 366 / 2000) loss: 31.509104
(Iteration 367 / 2000) loss: 29.165439
(Iteration 368 / 2000) loss: 30.389162
(Iteration 369 / 2000) loss: 31.576899
(Iteration 370 / 2000) loss: 32.668039
(Iteration 371 / 2000) loss: 35.993769
(Iteration 372 / 2000) loss: 30.142231
(Iteration 373 / 2000) loss: 33.116679
(Iteration 374 / 2000) loss: 32.243782
(Iteration 375 / 2000) loss: 30.585389
(Iteration 376 / 2000) loss: 29.813801
(Iteration 377 / 2000) loss: 31.494352
(Iteration 378 / 2000) loss: 30.985367
(Iteration 379 / 2000) loss: 30.225994
(Iteration 380 / 2000) loss: 29.844192
(Iteration 381 / 2000) loss: 28.149159
(Iteration 382 / 2000) loss: 32.335545
(Iteration 383 / 2000) loss: 28.354660
(Iteration 384 / 2000) loss: 29.358564
(Iteration 385 / 2000) loss: 29.245685
(Iteration 386 / 2000) loss: 30.549361
(Iteration 387 / 2000) loss: 36.330025
(Iteration 388 / 2000) loss: 32.029783
(Iteration 389 / 2000) loss: 33.431822
(Iteration 390 / 2000) loss: 27.908913
(Iteration 391 / 2000) loss: 26.035758
(Iteration 392 / 2000) loss: 33.140805
(Iteration 393 / 2000) loss: 36.233913
(Iteration 394 / 2000) loss: 27.628225
(Iteration 395 / 2000) loss: 32.408130
(Iteration 396 / 2000) loss: 34.318868
(Iteration 397 / 2000) loss: 29.377049
(Iteration 398 / 2000) loss: 30.298369
(Iteration 399 / 2000) loss: 29.263888
(Iteration 400 / 2000) loss: 32.352745
(Iteration 401 / 2000) loss: 29.194914
(Iteration 402 / 2000) loss: 30.234518
(Iteration 403 / 2000) loss: 29.673299
(Iteration 404 / 2000) loss: 33.926902
(Iteration 405 / 2000) loss: 30.473062
(Iteration 406 / 2000) loss: 28.162263
(Iteration 407 / 2000) loss: 34.526626
(Iteration 408 / 2000) loss: 30.868191
(Iteration 409 / 2000) loss: 28.283145
(Iteration 410 / 2000) loss: 30.641332
(Iteration 411 / 2000) loss: 27.617345
(Iteration 412 / 2000) loss: 32.757833
(Iteration 413 / 2000) loss: 29.325073
(Iteration 414 / 2000) loss: 35.336085
(Iteration 415 / 2000) loss: 29.699268
(Iteration 416 / 2000) loss: 30.623053
(Iteration 417 / 2000) loss: 27.967492
(Iteration 418 / 2000) loss: 35.265174
(Iteration 419 / 2000) loss: 30.987807
(Iteration 420 / 2000) loss: 29.367511
(Iteration 421 / 2000) loss: 33.906250
(Iteration 422 / 2000) loss: 32.156068
(Iteration 423 / 2000) loss: 29.936516
(Iteration 424 / 2000) loss: 29.534231
(Iteration 425 / 2000) loss: 29.998483
(Iteration 426 / 2000) loss: 32.082526
(Iteration 427 / 2000) loss: 28.228206
(Iteration 428 / 2000) loss: 30.173708
(Iteration 429 / 2000) loss: 30.182878
(Iteration 430 / 2000) loss: 31.369541
(Iteration 431 / 2000) loss: 31.933829
(Iteration 432 / 2000) loss: 31.532386
(Iteration 433 / 2000) loss: 29.882196
(Iteration 434 / 2000) loss: 26.375800
(Iteration 435 / 2000) loss: 32.071275
(Iteration 436 / 2000) loss: 31.100135
(Iteration 437 / 2000) loss: 29.688107
(Iteration 438 / 2000) loss: 31.045636
(Iteration 439 / 2000) loss: 34.279856
(Iteration 440 / 2000) loss: 31.220437
(Iteration 441 / 2000) loss: 30.138413
(Iteration 442 / 2000) loss: 29.066662
(Iteration 443 / 2000) loss: 33.737800
(Iteration 444 / 2000) loss: 29.758885
(Iteration 445 / 2000) loss: 29.722823
(Iteration 446 / 2000) loss: 33.156091
(Iteration 447 / 2000) loss: 30.930218
(Iteration 448 / 2000) loss: 31.374549
(Iteration 449 / 2000) loss: 28.242966
(Iteration 450 / 2000) loss: 26.967886
(Iteration 451 / 2000) loss: 28.284957
(Iteration 452 / 2000) loss: 27.833592
(Iteration 453 / 2000) loss: 31.988049
(Iteration 454 / 2000) loss: 31.489904
(Iteration 455 / 2000) loss: 27.333364
(Iteration 456 / 2000) loss: 30.712531
(Iteration 457 / 2000) loss: 28.605803
(Iteration 458 / 2000) loss: 30.682071
(Iteration 459 / 2000) loss: 28.612165
(Iteration 460 / 2000) loss: 33.465632
(Iteration 461 / 2000) loss: 31.537744
(Iteration 462 / 2000) loss: 29.006717
(Iteration 463 / 2000) loss: 30.873550
(Iteration 464 / 2000) loss: 28.408875
(Iteration 465 / 2000) loss: 27.624719
(Iteration 466 / 2000) loss: 29.901214
(Iteration 467 / 2000) loss: 27.894490
(Iteration 468 / 2000) loss: 33.602996
(Iteration 469 / 2000) loss: 33.902864
(Iteration 470 / 2000) loss: 29.264594
(Iteration 471 / 2000) loss: 32.387376
(Iteration 472 / 2000) loss: 29.301853
(Iteration 473 / 2000) loss: 26.296442
(Iteration 474 / 2000) loss: 29.542603
(Iteration 475 / 2000) loss: 34.419486
(Iteration 476 / 2000) loss: 33.870107
(Iteration 477 / 2000) loss: 29.039318
(Iteration 478 / 2000) loss: 30.574628
(Iteration 479 / 2000) loss: 31.885858
(Iteration 480 / 2000) loss: 29.670729
(Iteration 481 / 2000) loss: 25.457968
(Iteration 482 / 2000) loss: 29.466890
(Iteration 483 / 2000) loss: 28.412044
(Iteration 484 / 2000) loss: 28.967055
(Iteration 485 / 2000) loss: 31.689529
(Iteration 486 / 2000) loss: 31.271570
(Iteration 487 / 2000) loss: 27.805822
(Iteration 488 / 2000) loss: 30.532414
(Iteration 489 / 2000) loss: 28.777919
(Iteration 490 / 2000) loss: 30.116256
(Iteration 491 / 2000) loss: 29.373386
(Iteration 492 / 2000) loss: 31.052727
(Iteration 493 / 2000) loss: 30.721668
(Iteration 494 / 2000) loss: 28.251299
(Iteration 495 / 2000) loss: 28.009406
(Iteration 496 / 2000) loss: 27.738921
(Iteration 497 / 2000) loss: 28.280222
(Iteration 498 / 2000) loss: 30.529037
(Iteration 499 / 2000) loss: 27.829290
(Iteration 500 / 2000) loss: 28.224339
(Iteration 501 / 2000) loss: 31.755372
(Iteration 502 / 2000) loss: 30.891527
(Iteration 503 / 2000) loss: 28.815835
(Iteration 504 / 2000) loss: 28.861117
(Iteration 505 / 2000) loss: 28.810037
(Iteration 506 / 2000) loss: 28.495491
(Iteration 507 / 2000) loss: 28.883778
(Iteration 508 / 2000) loss: 29.603036
(Iteration 509 / 2000) loss: 29.646453
(Iteration 510 / 2000) loss: 27.803608
(Iteration 511 / 2000) loss: 32.010619
(Iteration 512 / 2000) loss: 28.069044
(Iteration 513 / 2000) loss: 32.393896
(Iteration 514 / 2000) loss: 33.005097
(Iteration 515 / 2000) loss: 28.238529
(Iteration 516 / 2000) loss: 27.698611
(Iteration 517 / 2000) loss: 34.803709
(Iteration 518 / 2000) loss: 27.758014
(Iteration 519 / 2000) loss: 30.563921
(Iteration 520 / 2000) loss: 30.560898
(Iteration 521 / 2000) loss: 28.043980
(Iteration 522 / 2000) loss: 28.558643
(Iteration 523 / 2000) loss: 29.355720
(Iteration 524 / 2000) loss: 31.859868
(Iteration 525 / 2000) loss: 28.143857
(Iteration 526 / 2000) loss: 28.908470
(Iteration 527 / 2000) loss: 31.115474
(Iteration 528 / 2000) loss: 30.903498
(Iteration 529 / 2000) loss: 32.280379
(Iteration 530 / 2000) loss: 29.340046
(Iteration 531 / 2000) loss: 26.866319
(Iteration 532 / 2000) loss: 26.179571
(Iteration 533 / 2000) loss: 28.770158
(Iteration 534 / 2000) loss: 28.879676
(Iteration 535 / 2000) loss: 34.012557
(Iteration 536 / 2000) loss: 31.346471
(Iteration 537 / 2000) loss: 32.224840
(Iteration 538 / 2000) loss: 28.996414
(Iteration 539 / 2000) loss: 30.999856
(Iteration 540 / 2000) loss: 27.406224
(Iteration 541 / 2000) loss: 28.722201
(Iteration 542 / 2000) loss: 29.315099
(Iteration 543 / 2000) loss: 30.634542
(Iteration 544 / 2000) loss: 29.999749
(Iteration 545 / 2000) loss: 28.663198
(Iteration 546 / 2000) loss: 27.581313
(Iteration 547 / 2000) loss: 27.979851
(Iteration 548 / 2000) loss: 31.433329
(Iteration 549 / 2000) loss: 32.463975
(Iteration 550 / 2000) loss: 27.560603
(Iteration 551 / 2000) loss: 29.098796
(Iteration 552 / 2000) loss: 29.702864
(Iteration 553 / 2000) loss: 29.960780
(Iteration 554 / 2000) loss: 29.164525
(Iteration 555 / 2000) loss: 27.835132
(Iteration 556 / 2000) loss: 25.817392
(Iteration 557 / 2000) loss: 26.390980
(Iteration 558 / 2000) loss: 31.992428
(Iteration 559 / 2000) loss: 26.536397
(Iteration 560 / 2000) loss: 29.007167
(Iteration 561 / 2000) loss: 27.782798
(Iteration 562 / 2000) loss: 28.149570
(Iteration 563 / 2000) loss: 28.688858
(Iteration 564 / 2000) loss: 29.595839
(Iteration 565 / 2000) loss: 28.847540
(Iteration 566 / 2000) loss: 29.850662
(Iteration 567 / 2000) loss: 31.607870
(Iteration 568 / 2000) loss: 30.345941
(Iteration 569 / 2000) loss: 30.686627
(Iteration 570 / 2000) loss: 24.972851
(Iteration 571 / 2000) loss: 29.400657
(Iteration 572 / 2000) loss: 25.979304
(Iteration 573 / 2000) loss: 33.717141
(Iteration 574 / 2000) loss: 32.531189
(Iteration 575 / 2000) loss: 31.603096
(Iteration 576 / 2000) loss: 29.735130
(Iteration 577 / 2000) loss: 29.192812
(Iteration 578 / 2000) loss: 31.839639
(Iteration 579 / 2000) loss: 27.877512
(Iteration 580 / 2000) loss: 32.621698
(Iteration 581 / 2000) loss: 27.825145
(Iteration 582 / 2000) loss: 31.162349
(Iteration 583 / 2000) loss: 27.620708
(Iteration 584 / 2000) loss: 30.296004
(Iteration 585 / 2000) loss: 26.714007
(Iteration 586 / 2000) loss: 31.914900
(Iteration 587 / 2000) loss: 31.104369
(Iteration 588 / 2000) loss: 30.850633
(Iteration 589 / 2000) loss: 27.964780
(Iteration 590 / 2000) loss: 28.831520
(Iteration 591 / 2000) loss: 27.790631
(Iteration 592 / 2000) loss: 27.690176
(Iteration 593 / 2000) loss: 28.294978
(Iteration 594 / 2000) loss: 26.431684
(Iteration 595 / 2000) loss: 26.646071
(Iteration 596 / 2000) loss: 30.454828
(Iteration 597 / 2000) loss: 29.866298
(Iteration 598 / 2000) loss: 30.826259
(Iteration 599 / 2000) loss: 28.008518
(Iteration 600 / 2000) loss: 29.112034
(Iteration 601 / 2000) loss: 25.759521
(Iteration 602 / 2000) loss: 23.301683
(Iteration 603 / 2000) loss: 29.125488
(Iteration 604 / 2000) loss: 28.913585
(Iteration 605 / 2000) loss: 26.191405
(Iteration 606 / 2000) loss: 28.940907
(Iteration 607 / 2000) loss: 32.154768
(Iteration 608 / 2000) loss: 32.738713
(Iteration 609 / 2000) loss: 28.475252
(Iteration 610 / 2000) loss: 26.648305
(Iteration 611 / 2000) loss: 25.550674
(Iteration 612 / 2000) loss: 26.726232
(Iteration 613 / 2000) loss: 29.937708
(Iteration 614 / 2000) loss: 32.100492
(Iteration 615 / 2000) loss: 27.381536
(Iteration 616 / 2000) loss: 30.235700
(Iteration 617 / 2000) loss: 30.046728
(Iteration 618 / 2000) loss: 29.125996
(Iteration 619 / 2000) loss: 28.385773
(Iteration 620 / 2000) loss: 26.342610
(Iteration 621 / 2000) loss: 29.114583
(Iteration 622 / 2000) loss: 29.762823
(Iteration 623 / 2000) loss: 26.393787
(Iteration 624 / 2000) loss: 32.755540
(Iteration 625 / 2000) loss: 27.078071
(Iteration 626 / 2000) loss: 26.290939
(Iteration 627 / 2000) loss: 28.278790
(Iteration 628 / 2000) loss: 28.861022
(Iteration 629 / 2000) loss: 28.826888
(Iteration 630 / 2000) loss: 28.116512
(Iteration 631 / 2000) loss: 29.726491
(Iteration 632 / 2000) loss: 29.208729
(Iteration 633 / 2000) loss: 27.623426
(Iteration 634 / 2000) loss: 28.252358
(Iteration 635 / 2000) loss: 27.347883
(Iteration 636 / 2000) loss: 27.411734
(Iteration 637 / 2000) loss: 26.676105
(Iteration 638 / 2000) loss: 27.434698
(Iteration 639 / 2000) loss: 29.084883
(Iteration 640 / 2000) loss: 29.640713
(Iteration 641 / 2000) loss: 27.136270
(Iteration 642 / 2000) loss: 26.976691
(Iteration 643 / 2000) loss: 29.532173
(Iteration 644 / 2000) loss: 27.009537
(Iteration 645 / 2000) loss: 27.882538
(Iteration 646 / 2000) loss: 31.473312
(Iteration 647 / 2000) loss: 25.494012
(Iteration 648 / 2000) loss: 25.429943
(Iteration 649 / 2000) loss: 30.397287
(Iteration 650 / 2000) loss: 25.812201
(Iteration 651 / 2000) loss: 30.074084
(Iteration 652 / 2000) loss: 28.856316
(Iteration 653 / 2000) loss: 31.866575
(Iteration 654 / 2000) loss: 28.717506
(Iteration 655 / 2000) loss: 31.770172
(Iteration 656 / 2000) loss: 29.140361
(Iteration 657 / 2000) loss: 30.109170
(Iteration 658 / 2000) loss: 28.042419
(Iteration 659 / 2000) loss: 25.454530
(Iteration 660 / 2000) loss: 30.749856
(Iteration 661 / 2000) loss: 26.341412
(Iteration 662 / 2000) loss: 26.830895
(Iteration 663 / 2000) loss: 26.236211
(Iteration 664 / 2000) loss: 29.023398
(Iteration 665 / 2000) loss: 26.012840
(Iteration 666 / 2000) loss: 29.113126
(Iteration 667 / 2000) loss: 28.705872
(Iteration 668 / 2000) loss: 25.851916
(Iteration 669 / 2000) loss: 31.654046
(Iteration 670 / 2000) loss: 32.024767
(Iteration 671 / 2000) loss: 29.386906
(Iteration 672 / 2000) loss: 28.639557
(Iteration 673 / 2000) loss: 28.440330
(Iteration 674 / 2000) loss: 29.315033
(Iteration 675 / 2000) loss: 25.341311
(Iteration 676 / 2000) loss: 25.603484
(Iteration 677 / 2000) loss: 27.775367
(Iteration 678 / 2000) loss: 28.353833
(Iteration 679 / 2000) loss: 27.403850
(Iteration 680 / 2000) loss: 32.465464
(Iteration 681 / 2000) loss: 31.395319
(Iteration 682 / 2000) loss: 30.440523
(Iteration 683 / 2000) loss: 25.926690
(Iteration 684 / 2000) loss: 27.502189
(Iteration 685 / 2000) loss: 32.718346
(Iteration 686 / 2000) loss: 29.717393
(Iteration 687 / 2000) loss: 24.067371
(Iteration 688 / 2000) loss: 25.998864
(Iteration 689 / 2000) loss: 28.975728
(Iteration 690 / 2000) loss: 29.461387
(Iteration 691 / 2000) loss: 27.766692
(Iteration 692 / 2000) loss: 27.959729
(Iteration 693 / 2000) loss: 27.338932
(Iteration 694 / 2000) loss: 28.359837
(Iteration 695 / 2000) loss: 28.083378
(Iteration 696 / 2000) loss: 28.787095
(Iteration 697 / 2000) loss: 26.264843
(Iteration 698 / 2000) loss: 32.338224
(Iteration 699 / 2000) loss: 24.674522
(Iteration 700 / 2000) loss: 29.082207
(Iteration 701 / 2000) loss: 28.042535
(Iteration 702 / 2000) loss: 28.762012
(Iteration 703 / 2000) loss: 28.478191
(Iteration 704 / 2000) loss: 30.932125
(Iteration 705 / 2000) loss: 27.251508
(Iteration 706 / 2000) loss: 29.474111
(Iteration 707 / 2000) loss: 29.441289
(Iteration 708 / 2000) loss: 27.873882
(Iteration 709 / 2000) loss: 29.441045
(Iteration 710 / 2000) loss: 28.378957
(Iteration 711 / 2000) loss: 28.525495
(Iteration 712 / 2000) loss: 26.500519
(Iteration 713 / 2000) loss: 28.403711
(Iteration 714 / 2000) loss: 25.815407
(Iteration 715 / 2000) loss: 26.901631
(Iteration 716 / 2000) loss: 31.215021
(Iteration 717 / 2000) loss: 26.857522
(Iteration 718 / 2000) loss: 26.847574
(Iteration 719 / 2000) loss: 28.787491
(Iteration 720 / 2000) loss: 26.821685
(Iteration 721 / 2000) loss: 31.333834
(Iteration 722 / 2000) loss: 28.066728
(Iteration 723 / 2000) loss: 24.996076
(Iteration 724 / 2000) loss: 26.677780
(Iteration 725 / 2000) loss: 27.372399
(Iteration 726 / 2000) loss: 27.803879
(Iteration 727 / 2000) loss: 28.814970
(Iteration 728 / 2000) loss: 28.726124
(Iteration 729 / 2000) loss: 26.447733
(Iteration 730 / 2000) loss: 28.547748
(Iteration 731 / 2000) loss: 30.611850
(Iteration 732 / 2000) loss: 26.312920
(Iteration 733 / 2000) loss: 25.840307
(Iteration 734 / 2000) loss: 27.423495
(Iteration 735 / 2000) loss: 28.154874
(Iteration 736 / 2000) loss: 24.256847
(Iteration 737 / 2000) loss: 26.554861
(Iteration 738 / 2000) loss: 27.366669
(Iteration 739 / 2000) loss: 28.893874
(Iteration 740 / 2000) loss: 26.920499
(Iteration 741 / 2000) loss: 30.829671
(Iteration 742 / 2000) loss: 28.621639
(Iteration 743 / 2000) loss: 27.290258
(Iteration 744 / 2000) loss: 26.271684
(Iteration 745 / 2000) loss: 25.418761
(Iteration 746 / 2000) loss: 28.749960
(Iteration 747 / 2000) loss: 26.979761
(Iteration 748 / 2000) loss: 28.933191
(Iteration 749 / 2000) loss: 29.504811
(Iteration 750 / 2000) loss: 25.293857
(Iteration 751 / 2000) loss: 28.581795
(Iteration 752 / 2000) loss: 24.980521
(Iteration 753 / 2000) loss: 26.428771
(Iteration 754 / 2000) loss: 27.279616
(Iteration 755 / 2000) loss: 25.265127
(Iteration 756 / 2000) loss: 29.954214
(Iteration 757 / 2000) loss: 26.411177
(Iteration 758 / 2000) loss: 28.399474
(Iteration 759 / 2000) loss: 22.909437
(Iteration 760 / 2000) loss: 29.481701
(Iteration 761 / 2000) loss: 29.230900
(Iteration 762 / 2000) loss: 26.923885
(Iteration 763 / 2000) loss: 28.957749
(Iteration 764 / 2000) loss: 28.174947
(Iteration 765 / 2000) loss: 25.581578
(Iteration 766 / 2000) loss: 29.487340
(Iteration 767 / 2000) loss: 30.534010
(Iteration 768 / 2000) loss: 24.653597
(Iteration 769 / 2000) loss: 31.194093
(Iteration 770 / 2000) loss: 29.161997
(Iteration 771 / 2000) loss: 29.445264
(Iteration 772 / 2000) loss: 24.596682
(Iteration 773 / 2000) loss: 26.638857
(Iteration 774 / 2000) loss: 26.651477
(Iteration 775 / 2000) loss: 27.124413
(Iteration 776 / 2000) loss: 28.054624
(Iteration 777 / 2000) loss: 32.963211
(Iteration 778 / 2000) loss: 31.036358
(Iteration 779 / 2000) loss: 24.421946
(Iteration 780 / 2000) loss: 26.438233
(Iteration 781 / 2000) loss: 28.288241
(Iteration 782 / 2000) loss: 29.796344
(Iteration 783 / 2000) loss: 26.978423
(Iteration 784 / 2000) loss: 27.285922
(Iteration 785 / 2000) loss: 26.702704
(Iteration 786 / 2000) loss: 27.702993
(Iteration 787 / 2000) loss: 26.064332
(Iteration 788 / 2000) loss: 26.654865
(Iteration 789 / 2000) loss: 27.902808
(Iteration 790 / 2000) loss: 24.072326
(Iteration 791 / 2000) loss: 27.253648
(Iteration 792 / 2000) loss: 26.384680
(Iteration 793 / 2000) loss: 26.693431
(Iteration 794 / 2000) loss: 28.748039
(Iteration 795 / 2000) loss: 25.007177
(Iteration 796 / 2000) loss: 25.633552
(Iteration 797 / 2000) loss: 26.294749
(Iteration 798 / 2000) loss: 27.763854
(Iteration 799 / 2000) loss: 25.867578
(Iteration 800 / 2000) loss: 29.851524
(Iteration 801 / 2000) loss: 25.644455
(Iteration 802 / 2000) loss: 29.112888
(Iteration 803 / 2000) loss: 31.743156
(Iteration 804 / 2000) loss: 25.385062
(Iteration 805 / 2000) loss: 25.189442
(Iteration 806 / 2000) loss: 30.224400
(Iteration 807 / 2000) loss: 24.999877
(Iteration 808 / 2000) loss: 30.220764
(Iteration 809 / 2000) loss: 28.784446
(Iteration 810 / 2000) loss: 29.954413
(Iteration 811 / 2000) loss: 27.690344
(Iteration 812 / 2000) loss: 27.794647
(Iteration 813 / 2000) loss: 30.292103
(Iteration 814 / 2000) loss: 26.394667
(Iteration 815 / 2000) loss: 27.095398
(Iteration 816 / 2000) loss: 25.996146
(Iteration 817 / 2000) loss: 28.487886
(Iteration 818 / 2000) loss: 28.595164
(Iteration 819 / 2000) loss: 28.100769
(Iteration 820 / 2000) loss: 28.467075
(Iteration 821 / 2000) loss: 26.098676
(Iteration 822 / 2000) loss: 27.994869
(Iteration 823 / 2000) loss: 28.692395
(Iteration 824 / 2000) loss: 27.737453
(Iteration 825 / 2000) loss: 23.913225
(Iteration 826 / 2000) loss: 25.931615
(Iteration 827 / 2000) loss: 28.006492
(Iteration 828 / 2000) loss: 28.773894
(Iteration 829 / 2000) loss: 23.032921
(Iteration 830 / 2000) loss: 29.151741
(Iteration 831 / 2000) loss: 26.347552
(Iteration 832 / 2000) loss: 27.249036
(Iteration 833 / 2000) loss: 24.932933
(Iteration 834 / 2000) loss: 28.782962
(Iteration 835 / 2000) loss: 27.578753
(Iteration 836 / 2000) loss: 25.155297
(Iteration 837 / 2000) loss: 24.968610
(Iteration 838 / 2000) loss: 27.081751
(Iteration 839 / 2000) loss: 28.740725
(Iteration 840 / 2000) loss: 26.993145
(Iteration 841 / 2000) loss: 28.907566
(Iteration 842 / 2000) loss: 24.135996
(Iteration 843 / 2000) loss: 28.952095
(Iteration 844 / 2000) loss: 31.120330
(Iteration 845 / 2000) loss: 25.818742
(Iteration 846 / 2000) loss: 25.814752
(Iteration 847 / 2000) loss: 31.569880
(Iteration 848 / 2000) loss: 31.679934
(Iteration 849 / 2000) loss: 26.007644
(Iteration 850 / 2000) loss: 28.455650
(Iteration 851 / 2000) loss: 27.057120
(Iteration 852 / 2000) loss: 31.437913
(Iteration 853 / 2000) loss: 26.451682
(Iteration 854 / 2000) loss: 26.236993
(Iteration 855 / 2000) loss: 30.753312
(Iteration 856 / 2000) loss: 22.288547
(Iteration 857 / 2000) loss: 25.955711
(Iteration 858 / 2000) loss: 24.243416
(Iteration 859 / 2000) loss: 27.284916
(Iteration 860 / 2000) loss: 29.670259
(Iteration 861 / 2000) loss: 24.778208
(Iteration 862 / 2000) loss: 28.982164
(Iteration 863 / 2000) loss: 21.688818
(Iteration 864 / 2000) loss: 30.074261
(Iteration 865 / 2000) loss: 24.679786
(Iteration 866 / 2000) loss: 27.253597
(Iteration 867 / 2000) loss: 26.967864
(Iteration 868 / 2000) loss: 25.439792
(Iteration 869 / 2000) loss: 25.596172
(Iteration 870 / 2000) loss: 27.858413
(Iteration 871 / 2000) loss: 30.577457
(Iteration 872 / 2000) loss: 25.774961
(Iteration 873 / 2000) loss: 26.188363
(Iteration 874 / 2000) loss: 28.974511
(Iteration 875 / 2000) loss: 26.112880
(Iteration 876 / 2000) loss: 25.374475
(Iteration 877 / 2000) loss: 24.571480
(Iteration 878 / 2000) loss: 28.420771
(Iteration 879 / 2000) loss: 24.783934
(Iteration 880 / 2000) loss: 23.763595
(Iteration 881 / 2000) loss: 29.062225
(Iteration 882 / 2000) loss: 26.701663
(Iteration 883 / 2000) loss: 27.089803
(Iteration 884 / 2000) loss: 26.807243
(Iteration 885 / 2000) loss: 26.136112
(Iteration 886 / 2000) loss: 30.582588
(Iteration 887 / 2000) loss: 28.038916
(Iteration 888 / 2000) loss: 24.058288
(Iteration 889 / 2000) loss: 27.994551
(Iteration 890 / 2000) loss: 25.222692
(Iteration 891 / 2000) loss: 31.693882
(Iteration 892 / 2000) loss: 29.471521
(Iteration 893 / 2000) loss: 27.291791
(Iteration 894 / 2000) loss: 25.089330
(Iteration 895 / 2000) loss: 30.392783
(Iteration 896 / 2000) loss: 25.139251
(Iteration 897 / 2000) loss: 25.290802
(Iteration 898 / 2000) loss: 28.075312
(Iteration 899 / 2000) loss: 27.223518
(Iteration 900 / 2000) loss: 24.610583
(Iteration 901 / 2000) loss: 29.901532
(Iteration 902 / 2000) loss: 27.566667
(Iteration 903 / 2000) loss: 30.637744
(Iteration 904 / 2000) loss: 26.855272
(Iteration 905 / 2000) loss: 26.083069
(Iteration 906 / 2000) loss: 26.838509
(Iteration 907 / 2000) loss: 27.570099
(Iteration 908 / 2000) loss: 26.290492
(Iteration 909 / 2000) loss: 27.393673
(Iteration 910 / 2000) loss: 27.693442
(Iteration 911 / 2000) loss: 28.938094
(Iteration 912 / 2000) loss: 27.986859
(Iteration 913 / 2000) loss: 28.146035
(Iteration 914 / 2000) loss: 28.300055
(Iteration 915 / 2000) loss: 27.481152
(Iteration 916 / 2000) loss: 25.656590
(Iteration 917 / 2000) loss: 24.615149
(Iteration 918 / 2000) loss: 26.470893
(Iteration 919 / 2000) loss: 29.440879
(Iteration 920 / 2000) loss: 26.466092
(Iteration 921 / 2000) loss: 30.097211
(Iteration 922 / 2000) loss: 26.989367
(Iteration 923 / 2000) loss: 26.478406
(Iteration 924 / 2000) loss: 26.756536
(Iteration 925 / 2000) loss: 30.869430
(Iteration 926 / 2000) loss: 29.629494
(Iteration 927 / 2000) loss: 28.806560
(Iteration 928 / 2000) loss: 28.196354
(Iteration 929 / 2000) loss: 25.533717
(Iteration 930 / 2000) loss: 27.595211
(Iteration 931 / 2000) loss: 24.772596
(Iteration 932 / 2000) loss: 27.571321
(Iteration 933 / 2000) loss: 25.848874
(Iteration 934 / 2000) loss: 26.268324
(Iteration 935 / 2000) loss: 28.844510
(Iteration 936 / 2000) loss: 23.779027
(Iteration 937 / 2000) loss: 27.329407
(Iteration 938 / 2000) loss: 23.362465
(Iteration 939 / 2000) loss: 24.985497
(Iteration 940 / 2000) loss: 28.397273
(Iteration 941 / 2000) loss: 27.318420
(Iteration 942 / 2000) loss: 24.161310
(Iteration 943 / 2000) loss: 28.222877
(Iteration 944 / 2000) loss: 28.584606
(Iteration 945 / 2000) loss: 26.458986
(Iteration 946 / 2000) loss: 27.976986
(Iteration 947 / 2000) loss: 24.948902
(Iteration 948 / 2000) loss: 29.284663
(Iteration 949 / 2000) loss: 26.311280
(Iteration 950 / 2000) loss: 25.009385
(Iteration 951 / 2000) loss: 28.943926
(Iteration 952 / 2000) loss: 23.435191
(Iteration 953 / 2000) loss: 24.932749
(Iteration 954 / 2000) loss: 28.327569
(Iteration 955 / 2000) loss: 26.292614
(Iteration 956 / 2000) loss: 24.498043
(Iteration 957 / 2000) loss: 28.011701
(Iteration 958 / 2000) loss: 26.427842
(Iteration 959 / 2000) loss: 27.659393
(Iteration 960 / 2000) loss: 25.246273
(Iteration 961 / 2000) loss: 23.002298
(Iteration 962 / 2000) loss: 25.613029
(Iteration 963 / 2000) loss: 26.344725
(Iteration 964 / 2000) loss: 26.702279
(Iteration 965 / 2000) loss: 26.281769
(Iteration 966 / 2000) loss: 26.972138
(Iteration 967 / 2000) loss: 26.382180
(Iteration 968 / 2000) loss: 23.143572
(Iteration 969 / 2000) loss: 25.566734
(Iteration 970 / 2000) loss: 28.655926
(Iteration 971 / 2000) loss: 28.923766
(Iteration 972 / 2000) loss: 26.737806
(Iteration 973 / 2000) loss: 28.262663
(Iteration 974 / 2000) loss: 25.575189
(Iteration 975 / 2000) loss: 31.828291
(Iteration 976 / 2000) loss: 26.902950
(Iteration 977 / 2000) loss: 30.922356
(Iteration 978 / 2000) loss: 25.692033
(Iteration 979 / 2000) loss: 25.975887
(Iteration 980 / 2000) loss: 27.069072
(Iteration 981 / 2000) loss: 29.370603
(Iteration 982 / 2000) loss: 25.349863
(Iteration 983 / 2000) loss: 24.981050
(Iteration 984 / 2000) loss: 31.632758
(Iteration 985 / 2000) loss: 24.875334
(Iteration 986 / 2000) loss: 27.746673
(Iteration 987 / 2000) loss: 25.483643
(Iteration 988 / 2000) loss: 25.045640
(Iteration 989 / 2000) loss: 28.063728
(Iteration 990 / 2000) loss: 26.769627
(Iteration 991 / 2000) loss: 27.547875
(Iteration 992 / 2000) loss: 27.650774
(Iteration 993 / 2000) loss: 27.346831
(Iteration 994 / 2000) loss: 24.072857
(Iteration 995 / 2000) loss: 24.063581
(Iteration 996 / 2000) loss: 24.407454
(Iteration 997 / 2000) loss: 23.296756
(Iteration 998 / 2000) loss: 25.222218
(Iteration 999 / 2000) loss: 28.453109
(Iteration 1000 / 2000) loss: 28.588210
(Iteration 1001 / 2000) loss: 27.261588
(Iteration 1002 / 2000) loss: 27.567649
(Iteration 1003 / 2000) loss: 25.781601
(Iteration 1004 / 2000) loss: 26.600506
(Iteration 1005 / 2000) loss: 25.015552
(Iteration 1006 / 2000) loss: 24.762985
(Iteration 1007 / 2000) loss: 21.883132
(Iteration 1008 / 2000) loss: 25.312756
(Iteration 1009 / 2000) loss: 31.598629
(Iteration 1010 / 2000) loss: 25.051389
(Iteration 1011 / 2000) loss: 26.084948
(Iteration 1012 / 2000) loss: 27.879236
(Iteration 1013 / 2000) loss: 26.868941
(Iteration 1014 / 2000) loss: 27.644716
(Iteration 1015 / 2000) loss: 25.509913
(Iteration 1016 / 2000) loss: 27.535586
(Iteration 1017 / 2000) loss: 31.289567
(Iteration 1018 / 2000) loss: 24.654876
(Iteration 1019 / 2000) loss: 26.484944
(Iteration 1020 / 2000) loss: 28.306393
(Iteration 1021 / 2000) loss: 26.680167
(Iteration 1022 / 2000) loss: 26.181402
(Iteration 1023 / 2000) loss: 26.275384
(Iteration 1024 / 2000) loss: 24.129580
(Iteration 1025 / 2000) loss: 26.146191
(Iteration 1026 / 2000) loss: 27.626897
(Iteration 1027 / 2000) loss: 23.613930
(Iteration 1028 / 2000) loss: 25.921319
(Iteration 1029 / 2000) loss: 25.798623
(Iteration 1030 / 2000) loss: 29.495621
(Iteration 1031 / 2000) loss: 24.517046
(Iteration 1032 / 2000) loss: 23.106100
(Iteration 1033 / 2000) loss: 24.791233
(Iteration 1034 / 2000) loss: 26.826113
(Iteration 1035 / 2000) loss: 27.479907
(Iteration 1036 / 2000) loss: 23.510217
(Iteration 1037 / 2000) loss: 20.424437
(Iteration 1038 / 2000) loss: 25.957196
(Iteration 1039 / 2000) loss: 23.605252
(Iteration 1040 / 2000) loss: 24.663255
(Iteration 1041 / 2000) loss: 23.169458
(Iteration 1042 / 2000) loss: 25.336147
(Iteration 1043 / 2000) loss: 25.437970
(Iteration 1044 / 2000) loss: 24.424158
(Iteration 1045 / 2000) loss: 23.771805
(Iteration 1046 / 2000) loss: 24.078706
(Iteration 1047 / 2000) loss: 27.166077
(Iteration 1048 / 2000) loss: 22.511880
(Iteration 1049 / 2000) loss: 25.914896
(Iteration 1050 / 2000) loss: 25.361351
(Iteration 1051 / 2000) loss: 26.655631
(Iteration 1052 / 2000) loss: 23.976805
(Iteration 1053 / 2000) loss: 26.439412
(Iteration 1054 / 2000) loss: 26.906625
(Iteration 1055 / 2000) loss: 28.247542
(Iteration 1056 / 2000) loss: 26.742020
(Iteration 1057 / 2000) loss: 28.211346
(Iteration 1058 / 2000) loss: 25.780138
(Iteration 1059 / 2000) loss: 26.750853
(Iteration 1060 / 2000) loss: 23.735673
(Iteration 1061 / 2000) loss: 25.202086
(Iteration 1062 / 2000) loss: 26.179038
(Iteration 1063 / 2000) loss: 26.783048
(Iteration 1064 / 2000) loss: 25.424837
(Iteration 1065 / 2000) loss: 28.526760
(Iteration 1066 / 2000) loss: 29.150875
(Iteration 1067 / 2000) loss: 23.236053
(Iteration 1068 / 2000) loss: 27.588107
(Iteration 1069 / 2000) loss: 26.118344
(Iteration 1070 / 2000) loss: 27.188885
(Iteration 1071 / 2000) loss: 22.345015
(Iteration 1072 / 2000) loss: 24.024886
(Iteration 1073 / 2000) loss: 27.974915
(Iteration 1074 / 2000) loss: 25.142411
(Iteration 1075 / 2000) loss: 24.701712
(Iteration 1076 / 2000) loss: 24.428148
(Iteration 1077 / 2000) loss: 24.148774
(Iteration 1078 / 2000) loss: 25.699379
(Iteration 1079 / 2000) loss: 27.526338
(Iteration 1080 / 2000) loss: 28.739992
(Iteration 1081 / 2000) loss: 24.002972
(Iteration 1082 / 2000) loss: 23.951679
(Iteration 1083 / 2000) loss: 25.327385
(Iteration 1084 / 2000) loss: 23.610454
(Iteration 1085 / 2000) loss: 26.458967
(Iteration 1086 / 2000) loss: 21.403525
(Iteration 1087 / 2000) loss: 24.146605
(Iteration 1088 / 2000) loss: 22.183386
(Iteration 1089 / 2000) loss: 24.606347
(Iteration 1090 / 2000) loss: 22.721796
(Iteration 1091 / 2000) loss: 27.382534
(Iteration 1092 / 2000) loss: 25.585642
(Iteration 1093 / 2000) loss: 27.129034
(Iteration 1094 / 2000) loss: 24.662007
(Iteration 1095 / 2000) loss: 26.706890
(Iteration 1096 / 2000) loss: 24.282865
(Iteration 1097 / 2000) loss: 25.838455
(Iteration 1098 / 2000) loss: 22.066717
(Iteration 1099 / 2000) loss: 24.593364
(Iteration 1100 / 2000) loss: 24.204539
(Iteration 1101 / 2000) loss: 26.235574
(Iteration 1102 / 2000) loss: 27.574628
(Iteration 1103 / 2000) loss: 23.477228
(Iteration 1104 / 2000) loss: 23.645136
(Iteration 1105 / 2000) loss: 25.521513
(Iteration 1106 / 2000) loss: 23.757002
(Iteration 1107 / 2000) loss: 28.618592
(Iteration 1108 / 2000) loss: 26.100336
(Iteration 1109 / 2000) loss: 25.420206
(Iteration 1110 / 2000) loss: 23.515843
(Iteration 1111 / 2000) loss: 24.288597
(Iteration 1112 / 2000) loss: 21.740183
(Iteration 1113 / 2000) loss: 27.096932
(Iteration 1114 / 2000) loss: 27.209829
(Iteration 1115 / 2000) loss: 23.618444
(Iteration 1116 / 2000) loss: 24.138016
(Iteration 1117 / 2000) loss: 27.864890
(Iteration 1118 / 2000) loss: 23.723639
(Iteration 1119 / 2000) loss: 25.393268
(Iteration 1120 / 2000) loss: 24.744400
(Iteration 1121 / 2000) loss: 25.373097
(Iteration 1122 / 2000) loss: 24.078903
(Iteration 1123 / 2000) loss: 22.062642
(Iteration 1124 / 2000) loss: 23.792917
(Iteration 1125 / 2000) loss: 23.585384
(Iteration 1126 / 2000) loss: 26.852754
(Iteration 1127 / 2000) loss: 25.823197
(Iteration 1128 / 2000) loss: 25.956165
(Iteration 1129 / 2000) loss: 23.808987
(Iteration 1130 / 2000) loss: 24.327518
(Iteration 1131 / 2000) loss: 24.406618
(Iteration 1132 / 2000) loss: 25.558841
(Iteration 1133 / 2000) loss: 23.038970
(Iteration 1134 / 2000) loss: 21.588659
(Iteration 1135 / 2000) loss: 23.372864
(Iteration 1136 / 2000) loss: 21.823551
(Iteration 1137 / 2000) loss: 24.590221
(Iteration 1138 / 2000) loss: 23.348236
(Iteration 1139 / 2000) loss: 27.902865
(Iteration 1140 / 2000) loss: 21.209865
(Iteration 1141 / 2000) loss: 26.288740
(Iteration 1142 / 2000) loss: 24.090810
(Iteration 1143 / 2000) loss: 26.665520
(Iteration 1144 / 2000) loss: 27.100436
(Iteration 1145 / 2000) loss: 26.983183
(Iteration 1146 / 2000) loss: 28.407209
(Iteration 1147 / 2000) loss: 27.123991
(Iteration 1148 / 2000) loss: 22.305070
(Iteration 1149 / 2000) loss: 25.777645
(Iteration 1150 / 2000) loss: 25.302225
(Iteration 1151 / 2000) loss: 23.502510
(Iteration 1152 / 2000) loss: 23.385555
(Iteration 1153 / 2000) loss: 22.579755
(Iteration 1154 / 2000) loss: 24.142599
(Iteration 1155 / 2000) loss: 23.995667
(Iteration 1156 / 2000) loss: 24.959802
(Iteration 1157 / 2000) loss: 25.669822
(Iteration 1158 / 2000) loss: 26.615843
(Iteration 1159 / 2000) loss: 24.802072
(Iteration 1160 / 2000) loss: 23.421019
(Iteration 1161 / 2000) loss: 25.217063
(Iteration 1162 / 2000) loss: 24.042579
(Iteration 1163 / 2000) loss: 22.157787
(Iteration 1164 / 2000) loss: 27.942843
(Iteration 1165 / 2000) loss: 26.652616
(Iteration 1166 / 2000) loss: 24.956848
(Iteration 1167 / 2000) loss: 22.335224
(Iteration 1168 / 2000) loss: 27.554877
(Iteration 1169 / 2000) loss: 26.345936
(Iteration 1170 / 2000) loss: 26.513572
(Iteration 1171 / 2000) loss: 24.618443
(Iteration 1172 / 2000) loss: 22.867994
(Iteration 1173 / 2000) loss: 27.849700
(Iteration 1174 / 2000) loss: 22.725151
(Iteration 1175 / 2000) loss: 25.135076
(Iteration 1176 / 2000) loss: 24.158279
(Iteration 1177 / 2000) loss: 24.504066
(Iteration 1178 / 2000) loss: 22.726508
(Iteration 1179 / 2000) loss: 23.842556
(Iteration 1180 / 2000) loss: 22.221203
(Iteration 1181 / 2000) loss: 22.832875
(Iteration 1182 / 2000) loss: 26.198520
(Iteration 1183 / 2000) loss: 24.011069
(Iteration 1184 / 2000) loss: 25.902291
(Iteration 1185 / 2000) loss: 25.540269
(Iteration 1186 / 2000) loss: 24.304269
(Iteration 1187 / 2000) loss: 28.221536
(Iteration 1188 / 2000) loss: 21.548525
(Iteration 1189 / 2000) loss: 27.225782
(Iteration 1190 / 2000) loss: 23.489018
(Iteration 1191 / 2000) loss: 25.342271
(Iteration 1192 / 2000) loss: 28.123936
(Iteration 1193 / 2000) loss: 25.815976
(Iteration 1194 / 2000) loss: 25.149996
(Iteration 1195 / 2000) loss: 24.833347
(Iteration 1196 / 2000) loss: 28.581717
(Iteration 1197 / 2000) loss: 28.779324
(Iteration 1198 / 2000) loss: 25.906147
(Iteration 1199 / 2000) loss: 22.310625
(Iteration 1200 / 2000) loss: 25.767195
(Iteration 1201 / 2000) loss: 21.980154
(Iteration 1202 / 2000) loss: 23.580493
(Iteration 1203 / 2000) loss: 24.365174
(Iteration 1204 / 2000) loss: 26.212405
(Iteration 1205 / 2000) loss: 27.240843
(Iteration 1206 / 2000) loss: 21.705710
(Iteration 1207 / 2000) loss: 26.727233
(Iteration 1208 / 2000) loss: 24.995620
(Iteration 1209 / 2000) loss: 27.260234
(Iteration 1210 / 2000) loss: 24.098512
(Iteration 1211 / 2000) loss: 27.428201
(Iteration 1212 / 2000) loss: 23.869046
(Iteration 1213 / 2000) loss: 25.417252
(Iteration 1214 / 2000) loss: 25.020214
(Iteration 1215 / 2000) loss: 23.648639
(Iteration 1216 / 2000) loss: 24.920659
(Iteration 1217 / 2000) loss: 22.120350
(Iteration 1218 / 2000) loss: 23.403156
(Iteration 1219 / 2000) loss: 26.733848
(Iteration 1220 / 2000) loss: 25.502418
(Iteration 1221 / 2000) loss: 24.508797
(Iteration 1222 / 2000) loss: 28.566997
(Iteration 1223 / 2000) loss: 25.028452
(Iteration 1224 / 2000) loss: 25.879369
(Iteration 1225 / 2000) loss: 25.748764
(Iteration 1226 / 2000) loss: 24.315035
(Iteration 1227 / 2000) loss: 24.335600
(Iteration 1228 / 2000) loss: 28.988712
(Iteration 1229 / 2000) loss: 22.871202
(Iteration 1230 / 2000) loss: 23.368079
(Iteration 1231 / 2000) loss: 24.242636
(Iteration 1232 / 2000) loss: 26.196116
(Iteration 1233 / 2000) loss: 25.951493
(Iteration 1234 / 2000) loss: 22.569568
(Iteration 1235 / 2000) loss: 24.325395
(Iteration 1236 / 2000) loss: 26.448265
(Iteration 1237 / 2000) loss: 22.392476
(Iteration 1238 / 2000) loss: 25.929997
(Iteration 1239 / 2000) loss: 24.655163
(Iteration 1240 / 2000) loss: 24.214411
(Iteration 1241 / 2000) loss: 19.756120
(Iteration 1242 / 2000) loss: 25.399149
(Iteration 1243 / 2000) loss: 26.074604
(Iteration 1244 / 2000) loss: 26.116745
(Iteration 1245 / 2000) loss: 27.964750
(Iteration 1246 / 2000) loss: 26.920702
(Iteration 1247 / 2000) loss: 24.168265
(Iteration 1248 / 2000) loss: 26.325466
(Iteration 1249 / 2000) loss: 29.037417
(Iteration 1250 / 2000) loss: 28.101683
(Iteration 1251 / 2000) loss: 25.628870
(Iteration 1252 / 2000) loss: 24.279210
(Iteration 1253 / 2000) loss: 28.389471
(Iteration 1254 / 2000) loss: 20.863433
(Iteration 1255 / 2000) loss: 30.738271
(Iteration 1256 / 2000) loss: 23.964076
(Iteration 1257 / 2000) loss: 26.394365
(Iteration 1258 / 2000) loss: 23.011264
(Iteration 1259 / 2000) loss: 28.185729
(Iteration 1260 / 2000) loss: 24.432165
(Iteration 1261 / 2000) loss: 22.898202
(Iteration 1262 / 2000) loss: 28.421515
(Iteration 1263 / 2000) loss: 22.789404
(Iteration 1264 / 2000) loss: 25.118641
(Iteration 1265 / 2000) loss: 26.863224
(Iteration 1266 / 2000) loss: 22.985071
(Iteration 1267 / 2000) loss: 22.903564
(Iteration 1268 / 2000) loss: 26.790159
(Iteration 1269 / 2000) loss: 29.244472
(Iteration 1270 / 2000) loss: 25.953510
(Iteration 1271 / 2000) loss: 24.500095
(Iteration 1272 / 2000) loss: 23.572271
(Iteration 1273 / 2000) loss: 27.268557
(Iteration 1274 / 2000) loss: 24.368670
(Iteration 1275 / 2000) loss: 24.323537
(Iteration 1276 / 2000) loss: 27.895793
(Iteration 1277 / 2000) loss: 27.086265
(Iteration 1278 / 2000) loss: 25.439721
(Iteration 1279 / 2000) loss: 24.417022
(Iteration 1280 / 2000) loss: 26.823553
(Iteration 1281 / 2000) loss: 22.157810
(Iteration 1282 / 2000) loss: 26.384955
(Iteration 1283 / 2000) loss: 25.016541
(Iteration 1284 / 2000) loss: 26.571310
(Iteration 1285 / 2000) loss: 23.349762
(Iteration 1286 / 2000) loss: 23.097508
(Iteration 1287 / 2000) loss: 22.340610
(Iteration 1288 / 2000) loss: 25.920064
(Iteration 1289 / 2000) loss: 24.856779
(Iteration 1290 / 2000) loss: 24.153332
(Iteration 1291 / 2000) loss: 23.207761
(Iteration 1292 / 2000) loss: 24.662323
(Iteration 1293 / 2000) loss: 20.954524
(Iteration 1294 / 2000) loss: 27.221392
(Iteration 1295 / 2000) loss: 25.642272
(Iteration 1296 / 2000) loss: 27.246960
(Iteration 1297 / 2000) loss: 25.805893
(Iteration 1298 / 2000) loss: 23.788664
(Iteration 1299 / 2000) loss: 24.753696
(Iteration 1300 / 2000) loss: 25.012162
(Iteration 1301 / 2000) loss: 22.725250
(Iteration 1302 / 2000) loss: 23.434916
(Iteration 1303 / 2000) loss: 20.034826
(Iteration 1304 / 2000) loss: 24.273698
(Iteration 1305 / 2000) loss: 22.972392
(Iteration 1306 / 2000) loss: 23.220994
(Iteration 1307 / 2000) loss: 27.128128
(Iteration 1308 / 2000) loss: 26.995792
(Iteration 1309 / 2000) loss: 24.893352
(Iteration 1310 / 2000) loss: 22.422341
(Iteration 1311 / 2000) loss: 24.455729
(Iteration 1312 / 2000) loss: 26.452529
(Iteration 1313 / 2000) loss: 21.917169
(Iteration 1314 / 2000) loss: 23.598185
(Iteration 1315 / 2000) loss: 25.025414
(Iteration 1316 / 2000) loss: 21.032467
(Iteration 1317 / 2000) loss: 26.735510
(Iteration 1318 / 2000) loss: 27.296643
(Iteration 1319 / 2000) loss: 23.483864
(Iteration 1320 / 2000) loss: 24.020211
(Iteration 1321 / 2000) loss: 26.771704
(Iteration 1322 / 2000) loss: 26.909517
(Iteration 1323 / 2000) loss: 21.711554
(Iteration 1324 / 2000) loss: 25.907662
(Iteration 1325 / 2000) loss: 20.563295
(Iteration 1326 / 2000) loss: 25.772413
(Iteration 1327 / 2000) loss: 27.795268
(Iteration 1328 / 2000) loss: 23.078483
(Iteration 1329 / 2000) loss: 22.170762
(Iteration 1330 / 2000) loss: 21.338833
(Iteration 1331 / 2000) loss: 22.441443
(Iteration 1332 / 2000) loss: 27.195460
(Iteration 1333 / 2000) loss: 26.594735
(Iteration 1334 / 2000) loss: 26.919560
(Iteration 1335 / 2000) loss: 28.720896
(Iteration 1336 / 2000) loss: 23.514236
(Iteration 1337 / 2000) loss: 25.610731
(Iteration 1338 / 2000) loss: 24.684904
(Iteration 1339 / 2000) loss: 25.073234
(Iteration 1340 / 2000) loss: 26.966992
(Iteration 1341 / 2000) loss: 23.345093
(Iteration 1342 / 2000) loss: 25.903342
(Iteration 1343 / 2000) loss: 25.744331
(Iteration 1344 / 2000) loss: 27.450198
(Iteration 1345 / 2000) loss: 25.624911
(Iteration 1346 / 2000) loss: 21.445845
(Iteration 1347 / 2000) loss: 24.982445
(Iteration 1348 / 2000) loss: 20.767899
(Iteration 1349 / 2000) loss: 25.420541
(Iteration 1350 / 2000) loss: 24.510907
(Iteration 1351 / 2000) loss: 25.177485
(Iteration 1352 / 2000) loss: 19.117364
(Iteration 1353 / 2000) loss: 24.096332
(Iteration 1354 / 2000) loss: 25.170756
(Iteration 1355 / 2000) loss: 28.508134
(Iteration 1356 / 2000) loss: 22.800403
(Iteration 1357 / 2000) loss: 21.896366
(Iteration 1358 / 2000) loss: 27.173390
(Iteration 1359 / 2000) loss: 23.956304
(Iteration 1360 / 2000) loss: 23.873304
(Iteration 1361 / 2000) loss: 25.673091
(Iteration 1362 / 2000) loss: 21.298296
(Iteration 1363 / 2000) loss: 24.315121
(Iteration 1364 / 2000) loss: 25.417123
(Iteration 1365 / 2000) loss: 25.791739
(Iteration 1366 / 2000) loss: 28.910760
(Iteration 1367 / 2000) loss: 19.757947
(Iteration 1368 / 2000) loss: 28.089504
(Iteration 1369 / 2000) loss: 27.581934
(Iteration 1370 / 2000) loss: 25.598485
(Iteration 1371 / 2000) loss: 24.909864
(Iteration 1372 / 2000) loss: 27.793807
(Iteration 1373 / 2000) loss: 26.961597
(Iteration 1374 / 2000) loss: 23.695717
(Iteration 1375 / 2000) loss: 25.690136
(Iteration 1376 / 2000) loss: 25.133001
(Iteration 1377 / 2000) loss: 26.539471
(Iteration 1378 / 2000) loss: 21.608527
(Iteration 1379 / 2000) loss: 24.651771
(Iteration 1380 / 2000) loss: 25.351998
(Iteration 1381 / 2000) loss: 25.602119
(Iteration 1382 / 2000) loss: 20.956756
(Iteration 1383 / 2000) loss: 27.044700
(Iteration 1384 / 2000) loss: 23.684446
(Iteration 1385 / 2000) loss: 24.007422
(Iteration 1386 / 2000) loss: 26.224486
(Iteration 1387 / 2000) loss: 24.091833
(Iteration 1388 / 2000) loss: 22.804724
(Iteration 1389 / 2000) loss: 26.333847
(Iteration 1390 / 2000) loss: 22.368797
(Iteration 1391 / 2000) loss: 31.096213
(Iteration 1392 / 2000) loss: 21.199399
(Iteration 1393 / 2000) loss: 27.029800
(Iteration 1394 / 2000) loss: 26.728485
(Iteration 1395 / 2000) loss: 22.530518
(Iteration 1396 / 2000) loss: 26.262471
(Iteration 1397 / 2000) loss: 22.182511
(Iteration 1398 / 2000) loss: 22.660986
(Iteration 1399 / 2000) loss: 23.354062
(Iteration 1400 / 2000) loss: 23.683805
(Iteration 1401 / 2000) loss: 24.522752
(Iteration 1402 / 2000) loss: 27.099434
(Iteration 1403 / 2000) loss: 26.447373
(Iteration 1404 / 2000) loss: 21.024984
(Iteration 1405 / 2000) loss: 23.553493
(Iteration 1406 / 2000) loss: 22.467736
(Iteration 1407 / 2000) loss: 25.487796
(Iteration 1408 / 2000) loss: 21.214106
(Iteration 1409 / 2000) loss: 24.809777
(Iteration 1410 / 2000) loss: 21.845171
(Iteration 1411 / 2000) loss: 24.948856
(Iteration 1412 / 2000) loss: 23.667809
(Iteration 1413 / 2000) loss: 28.275034
(Iteration 1414 / 2000) loss: 25.484515
(Iteration 1415 / 2000) loss: 21.443425
(Iteration 1416 / 2000) loss: 23.269724
(Iteration 1417 / 2000) loss: 22.509507
(Iteration 1418 / 2000) loss: 25.178126
(Iteration 1419 / 2000) loss: 23.642907
(Iteration 1420 / 2000) loss: 24.548570
(Iteration 1421 / 2000) loss: 23.191471
(Iteration 1422 / 2000) loss: 26.102352
(Iteration 1423 / 2000) loss: 23.508140
(Iteration 1424 / 2000) loss: 24.127591
(Iteration 1425 / 2000) loss: 22.591851
(Iteration 1426 / 2000) loss: 29.631039
(Iteration 1427 / 2000) loss: 23.675992
(Iteration 1428 / 2000) loss: 21.142325
(Iteration 1429 / 2000) loss: 22.142162
(Iteration 1430 / 2000) loss: 23.864753
(Iteration 1431 / 2000) loss: 27.723673
(Iteration 1432 / 2000) loss: 26.287701
(Iteration 1433 / 2000) loss: 24.082257
(Iteration 1434 / 2000) loss: 22.836958
(Iteration 1435 / 2000) loss: 21.525031
(Iteration 1436 / 2000) loss: 24.920727
(Iteration 1437 / 2000) loss: 26.791023
(Iteration 1438 / 2000) loss: 19.448319
(Iteration 1439 / 2000) loss: 23.589742
(Iteration 1440 / 2000) loss: 22.375965
(Iteration 1441 / 2000) loss: 25.338963
(Iteration 1442 / 2000) loss: 23.221514
(Iteration 1443 / 2000) loss: 21.297734
(Iteration 1444 / 2000) loss: 23.534230
(Iteration 1445 / 2000) loss: 23.870581
(Iteration 1446 / 2000) loss: 24.517740
(Iteration 1447 / 2000) loss: 26.286068
(Iteration 1448 / 2000) loss: 22.083762
(Iteration 1449 / 2000) loss: 21.986144
(Iteration 1450 / 2000) loss: 22.866761
(Iteration 1451 / 2000) loss: 28.046568
(Iteration 1452 / 2000) loss: 26.708998
(Iteration 1453 / 2000) loss: 22.632209
(Iteration 1454 / 2000) loss: 23.059684
(Iteration 1455 / 2000) loss: 22.191113
(Iteration 1456 / 2000) loss: 25.981371
(Iteration 1457 / 2000) loss: 22.937400
(Iteration 1458 / 2000) loss: 23.758930
(Iteration 1459 / 2000) loss: 26.841879
(Iteration 1460 / 2000) loss: 26.395369
(Iteration 1461 / 2000) loss: 25.218799
(Iteration 1462 / 2000) loss: 24.671441
(Iteration 1463 / 2000) loss: 26.582940
(Iteration 1464 / 2000) loss: 22.580776
(Iteration 1465 / 2000) loss: 25.090250
(Iteration 1466 / 2000) loss: 23.403810
(Iteration 1467 / 2000) loss: 26.415204
(Iteration 1468 / 2000) loss: 23.291348
(Iteration 1469 / 2000) loss: 25.441331
(Iteration 1470 / 2000) loss: 21.546579
(Iteration 1471 / 2000) loss: 22.823214
(Iteration 1472 / 2000) loss: 21.472828
(Iteration 1473 / 2000) loss: 22.907678
(Iteration 1474 / 2000) loss: 22.136526
(Iteration 1475 / 2000) loss: 24.310098
(Iteration 1476 / 2000) loss: 24.562716
(Iteration 1477 / 2000) loss: 22.661746
(Iteration 1478 / 2000) loss: 24.148640
(Iteration 1479 / 2000) loss: 23.044548
(Iteration 1480 / 2000) loss: 24.192827
(Iteration 1481 / 2000) loss: 25.814272
(Iteration 1482 / 2000) loss: 25.956170
(Iteration 1483 / 2000) loss: 21.456064
(Iteration 1484 / 2000) loss: 22.851555
(Iteration 1485 / 2000) loss: 22.869496
(Iteration 1486 / 2000) loss: 19.886116
(Iteration 1487 / 2000) loss: 21.855391
(Iteration 1488 / 2000) loss: 21.534876
(Iteration 1489 / 2000) loss: 22.701461
(Iteration 1490 / 2000) loss: 19.718452
(Iteration 1491 / 2000) loss: 21.918370
(Iteration 1492 / 2000) loss: 24.519247
(Iteration 1493 / 2000) loss: 23.505775
(Iteration 1494 / 2000) loss: 23.013972
(Iteration 1495 / 2000) loss: 19.868687
(Iteration 1496 / 2000) loss: 22.336467
(Iteration 1497 / 2000) loss: 25.691839
(Iteration 1498 / 2000) loss: 22.641018
(Iteration 1499 / 2000) loss: 24.150045
(Iteration 1500 / 2000) loss: 21.133004
(Iteration 1501 / 2000) loss: 23.830390
(Iteration 1502 / 2000) loss: 27.180692
(Iteration 1503 / 2000) loss: 23.684334
(Iteration 1504 / 2000) loss: 25.340621
(Iteration 1505 / 2000) loss: 23.644175
(Iteration 1506 / 2000) loss: 24.427611
(Iteration 1507 / 2000) loss: 24.809356
(Iteration 1508 / 2000) loss: 21.766342
(Iteration 1509 / 2000) loss: 21.026485
(Iteration 1510 / 2000) loss: 24.366702
(Iteration 1511 / 2000) loss: 24.626795
(Iteration 1512 / 2000) loss: 22.141828
(Iteration 1513 / 2000) loss: 24.909347
(Iteration 1514 / 2000) loss: 24.197167
(Iteration 1515 / 2000) loss: 18.934737
(Iteration 1516 / 2000) loss: 23.946113
(Iteration 1517 / 2000) loss: 27.519390
(Iteration 1518 / 2000) loss: 22.085042
(Iteration 1519 / 2000) loss: 23.265900
(Iteration 1520 / 2000) loss: 23.061189
(Iteration 1521 / 2000) loss: 23.159794
(Iteration 1522 / 2000) loss: 25.475460
(Iteration 1523 / 2000) loss: 21.152460
(Iteration 1524 / 2000) loss: 23.051852
(Iteration 1525 / 2000) loss: 27.319828
(Iteration 1526 / 2000) loss: 19.793336
(Iteration 1527 / 2000) loss: 25.984097
(Iteration 1528 / 2000) loss: 22.633950
(Iteration 1529 / 2000) loss: 22.663642
(Iteration 1530 / 2000) loss: 22.948860
(Iteration 1531 / 2000) loss: 22.810111
(Iteration 1532 / 2000) loss: 21.645358
(Iteration 1533 / 2000) loss: 26.008635
(Iteration 1534 / 2000) loss: 23.793042
(Iteration 1535 / 2000) loss: 21.801697
(Iteration 1536 / 2000) loss: 23.194571
(Iteration 1537 / 2000) loss: 23.077611
(Iteration 1538 / 2000) loss: 25.921327
(Iteration 1539 / 2000) loss: 25.141669
(Iteration 1540 / 2000) loss: 22.020129
(Iteration 1541 / 2000) loss: 22.812373
(Iteration 1542 / 2000) loss: 22.093407
(Iteration 1543 / 2000) loss: 24.255243
(Iteration 1544 / 2000) loss: 24.970891
(Iteration 1545 / 2000) loss: 24.157932
(Iteration 1546 / 2000) loss: 22.636549
(Iteration 1547 / 2000) loss: 22.229312
(Iteration 1548 / 2000) loss: 21.006100
(Iteration 1549 / 2000) loss: 23.180870
(Iteration 1550 / 2000) loss: 21.064525
(Iteration 1551 / 2000) loss: 23.190144
(Iteration 1552 / 2000) loss: 21.419418
(Iteration 1553 / 2000) loss: 25.318244
(Iteration 1554 / 2000) loss: 23.712348
(Iteration 1555 / 2000) loss: 25.157044
(Iteration 1556 / 2000) loss: 23.380242
(Iteration 1557 / 2000) loss: 25.590390
(Iteration 1558 / 2000) loss: 25.477091
(Iteration 1559 / 2000) loss: 26.203009
(Iteration 1560 / 2000) loss: 22.478253
(Iteration 1561 / 2000) loss: 21.950271
(Iteration 1562 / 2000) loss: 24.645745
(Iteration 1563 / 2000) loss: 22.634547
(Iteration 1564 / 2000) loss: 25.888325
(Iteration 1565 / 2000) loss: 20.197599
(Iteration 1566 / 2000) loss: 21.979197
(Iteration 1567 / 2000) loss: 20.380777
(Iteration 1568 / 2000) loss: 20.331868
(Iteration 1569 / 2000) loss: 22.980576
(Iteration 1570 / 2000) loss: 22.302553
(Iteration 1571 / 2000) loss: 20.789401
(Iteration 1572 / 2000) loss: 23.774094
(Iteration 1573 / 2000) loss: 22.877482
(Iteration 1574 / 2000) loss: 21.274390
(Iteration 1575 / 2000) loss: 20.637027
(Iteration 1576 / 2000) loss: 25.070315
(Iteration 1577 / 2000) loss: 22.153120
(Iteration 1578 / 2000) loss: 20.432604
(Iteration 1579 / 2000) loss: 26.086017
(Iteration 1580 / 2000) loss: 27.796283
(Iteration 1581 / 2000) loss: 23.935130
(Iteration 1582 / 2000) loss: 24.243764
(Iteration 1583 / 2000) loss: 26.005361
(Iteration 1584 / 2000) loss: 21.585895
(Iteration 1585 / 2000) loss: 20.041025
(Iteration 1586 / 2000) loss: 22.515765
(Iteration 1587 / 2000) loss: 21.303428
(Iteration 1588 / 2000) loss: 19.156181
(Iteration 1589 / 2000) loss: 25.167040
(Iteration 1590 / 2000) loss: 21.641782
(Iteration 1591 / 2000) loss: 25.446581
(Iteration 1592 / 2000) loss: 21.649999
(Iteration 1593 / 2000) loss: 25.356873
(Iteration 1594 / 2000) loss: 22.120884
(Iteration 1595 / 2000) loss: 24.777501
(Iteration 1596 / 2000) loss: 24.825576
(Iteration 1597 / 2000) loss: 24.905172
(Iteration 1598 / 2000) loss: 20.913417
(Iteration 1599 / 2000) loss: 24.266098
(Iteration 1600 / 2000) loss: 23.569884
(Iteration 1601 / 2000) loss: 22.450089
(Iteration 1602 / 2000) loss: 22.346361
(Iteration 1603 / 2000) loss: 25.468004
(Iteration 1604 / 2000) loss: 21.721452
(Iteration 1605 / 2000) loss: 22.807885
(Iteration 1606 / 2000) loss: 21.363168
(Iteration 1607 / 2000) loss: 23.318931
(Iteration 1608 / 2000) loss: 23.050137
(Iteration 1609 / 2000) loss: 21.641356
(Iteration 1610 / 2000) loss: 22.193292
(Iteration 1611 / 2000) loss: 22.874602
(Iteration 1612 / 2000) loss: 25.048292
(Iteration 1613 / 2000) loss: 24.563970
(Iteration 1614 / 2000) loss: 23.775347
(Iteration 1615 / 2000) loss: 20.750406
(Iteration 1616 / 2000) loss: 24.461780
(Iteration 1617 / 2000) loss: 24.314096
(Iteration 1618 / 2000) loss: 23.312131
(Iteration 1619 / 2000) loss: 21.321205
(Iteration 1620 / 2000) loss: 22.482504
(Iteration 1621 / 2000) loss: 22.435922
(Iteration 1622 / 2000) loss: 21.973121
(Iteration 1623 / 2000) loss: 20.012858
(Iteration 1624 / 2000) loss: 22.986115
(Iteration 1625 / 2000) loss: 22.910851
(Iteration 1626 / 2000) loss: 22.258956
(Iteration 1627 / 2000) loss: 22.193450
(Iteration 1628 / 2000) loss: 22.583380
(Iteration 1629 / 2000) loss: 21.652369
(Iteration 1630 / 2000) loss: 24.844997
(Iteration 1631 / 2000) loss: 21.644182
(Iteration 1632 / 2000) loss: 23.048442
(Iteration 1633 / 2000) loss: 19.690835
(Iteration 1634 / 2000) loss: 23.393121
(Iteration 1635 / 2000) loss: 20.834758
(Iteration 1636 / 2000) loss: 23.726716
(Iteration 1637 / 2000) loss: 19.872807
(Iteration 1638 / 2000) loss: 25.041268
(Iteration 1639 / 2000) loss: 25.772813
(Iteration 1640 / 2000) loss: 22.684606
(Iteration 1641 / 2000) loss: 24.782589
(Iteration 1642 / 2000) loss: 21.407744
(Iteration 1643 / 2000) loss: 23.133376
(Iteration 1644 / 2000) loss: 25.677870
(Iteration 1645 / 2000) loss: 22.558352
(Iteration 1646 / 2000) loss: 23.593245
(Iteration 1647 / 2000) loss: 26.091226
(Iteration 1648 / 2000) loss: 23.282335
(Iteration 1649 / 2000) loss: 20.315354
(Iteration 1650 / 2000) loss: 23.853788
(Iteration 1651 / 2000) loss: 26.546207
(Iteration 1652 / 2000) loss: 22.775372
(Iteration 1653 / 2000) loss: 22.531886
(Iteration 1654 / 2000) loss: 23.105267
(Iteration 1655 / 2000) loss: 19.645694
(Iteration 1656 / 2000) loss: 20.629694
(Iteration 1657 / 2000) loss: 23.087392
(Iteration 1658 / 2000) loss: 25.757952
(Iteration 1659 / 2000) loss: 21.030283
(Iteration 1660 / 2000) loss: 26.684851
(Iteration 1661 / 2000) loss: 25.773171
(Iteration 1662 / 2000) loss: 22.555075
(Iteration 1663 / 2000) loss: 24.445283
(Iteration 1664 / 2000) loss: 20.116101
(Iteration 1665 / 2000) loss: 23.518166
(Iteration 1666 / 2000) loss: 27.482090
(Iteration 1667 / 2000) loss: 22.559103
(Iteration 1668 / 2000) loss: 25.461522
(Iteration 1669 / 2000) loss: 21.326597
(Iteration 1670 / 2000) loss: 21.527796
(Iteration 1671 / 2000) loss: 24.291668
(Iteration 1672 / 2000) loss: 22.044947
(Iteration 1673 / 2000) loss: 21.423922
(Iteration 1674 / 2000) loss: 21.311655
(Iteration 1675 / 2000) loss: 24.353749
(Iteration 1676 / 2000) loss: 24.667957
(Iteration 1677 / 2000) loss: 22.026329
(Iteration 1678 / 2000) loss: 24.707507
(Iteration 1679 / 2000) loss: 24.431852
(Iteration 1680 / 2000) loss: 21.820296
(Iteration 1681 / 2000) loss: 24.999355
(Iteration 1682 / 2000) loss: 24.619063
(Iteration 1683 / 2000) loss: 25.277358
(Iteration 1684 / 2000) loss: 25.223813
(Iteration 1685 / 2000) loss: 20.580917
(Iteration 1686 / 2000) loss: 22.266933
(Iteration 1687 / 2000) loss: 20.992912
(Iteration 1688 / 2000) loss: 25.875691
(Iteration 1689 / 2000) loss: 22.460154
(Iteration 1690 / 2000) loss: 23.339040
(Iteration 1691 / 2000) loss: 21.028499
(Iteration 1692 / 2000) loss: 21.999006
(Iteration 1693 / 2000) loss: 23.631529
(Iteration 1694 / 2000) loss: 22.287241
(Iteration 1695 / 2000) loss: 21.397750
(Iteration 1696 / 2000) loss: 21.357007
(Iteration 1697 / 2000) loss: 22.680145
(Iteration 1698 / 2000) loss: 21.847909
(Iteration 1699 / 2000) loss: 22.105310
(Iteration 1700 / 2000) loss: 23.184481
(Iteration 1701 / 2000) loss: 22.975046
(Iteration 1702 / 2000) loss: 22.846959
(Iteration 1703 / 2000) loss: 24.220193
(Iteration 1704 / 2000) loss: 24.069204
(Iteration 1705 / 2000) loss: 21.740525
(Iteration 1706 / 2000) loss: 23.001037
(Iteration 1707 / 2000) loss: 26.301966
(Iteration 1708 / 2000) loss: 23.221991
(Iteration 1709 / 2000) loss: 24.111051
(Iteration 1710 / 2000) loss: 24.069533
(Iteration 1711 / 2000) loss: 18.602880
(Iteration 1712 / 2000) loss: 24.371294
(Iteration 1713 / 2000) loss: 26.429083
(Iteration 1714 / 2000) loss: 25.589780
(Iteration 1715 / 2000) loss: 22.368203
(Iteration 1716 / 2000) loss: 19.753293
(Iteration 1717 / 2000) loss: 20.139103
(Iteration 1718 / 2000) loss: 23.235651
(Iteration 1719 / 2000) loss: 25.114780
(Iteration 1720 / 2000) loss: 28.567757
(Iteration 1721 / 2000) loss: 23.353418
(Iteration 1722 / 2000) loss: 23.235364
(Iteration 1723 / 2000) loss: 21.927765
(Iteration 1724 / 2000) loss: 22.702755
(Iteration 1725 / 2000) loss: 23.782636
(Iteration 1726 / 2000) loss: 22.875927
(Iteration 1727 / 2000) loss: 22.089354
(Iteration 1728 / 2000) loss: 25.231356
(Iteration 1729 / 2000) loss: 22.068732
(Iteration 1730 / 2000) loss: 20.798636
(Iteration 1731 / 2000) loss: 24.002694
(Iteration 1732 / 2000) loss: 23.536411
(Iteration 1733 / 2000) loss: 24.417722
(Iteration 1734 / 2000) loss: 21.774420
(Iteration 1735 / 2000) loss: 21.787926
(Iteration 1736 / 2000) loss: 20.550872
(Iteration 1737 / 2000) loss: 22.904871
(Iteration 1738 / 2000) loss: 21.260637
(Iteration 1739 / 2000) loss: 20.309491
(Iteration 1740 / 2000) loss: 20.027294
(Iteration 1741 / 2000) loss: 21.134384
(Iteration 1742 / 2000) loss: 23.220460
(Iteration 1743 / 2000) loss: 22.434686
(Iteration 1744 / 2000) loss: 22.582194
(Iteration 1745 / 2000) loss: 22.250390
(Iteration 1746 / 2000) loss: 21.935453
(Iteration 1747 / 2000) loss: 22.655585
(Iteration 1748 / 2000) loss: 23.041471
(Iteration 1749 / 2000) loss: 24.679240
(Iteration 1750 / 2000) loss: 23.955791
(Iteration 1751 / 2000) loss: 23.698404
(Iteration 1752 / 2000) loss: 24.860086
(Iteration 1753 / 2000) loss: 22.920053
(Iteration 1754 / 2000) loss: 22.554974
(Iteration 1755 / 2000) loss: 25.370224
(Iteration 1756 / 2000) loss: 20.628359
(Iteration 1757 / 2000) loss: 25.405234
(Iteration 1758 / 2000) loss: 20.986866
(Iteration 1759 / 2000) loss: 23.360913
(Iteration 1760 / 2000) loss: 21.608336
(Iteration 1761 / 2000) loss: 24.114160
(Iteration 1762 / 2000) loss: 20.992294
(Iteration 1763 / 2000) loss: 24.756728
(Iteration 1764 / 2000) loss: 21.163161
(Iteration 1765 / 2000) loss: 24.048984
(Iteration 1766 / 2000) loss: 27.114108
(Iteration 1767 / 2000) loss: 23.520527
(Iteration 1768 / 2000) loss: 19.867550
(Iteration 1769 / 2000) loss: 21.164872
(Iteration 1770 / 2000) loss: 19.409120
(Iteration 1771 / 2000) loss: 23.099109
(Iteration 1772 / 2000) loss: 21.095711
(Iteration 1773 / 2000) loss: 22.390782
(Iteration 1774 / 2000) loss: 20.767173
(Iteration 1775 / 2000) loss: 21.738760
(Iteration 1776 / 2000) loss: 21.776635
(Iteration 1777 / 2000) loss: 19.602204
(Iteration 1778 / 2000) loss: 22.060161
(Iteration 1779 / 2000) loss: 21.554491
(Iteration 1780 / 2000) loss: 22.329255
(Iteration 1781 / 2000) loss: 23.788196
(Iteration 1782 / 2000) loss: 25.366220
(Iteration 1783 / 2000) loss: 23.479777
(Iteration 1784 / 2000) loss: 22.229720
(Iteration 1785 / 2000) loss: 21.493983
(Iteration 1786 / 2000) loss: 22.536950
(Iteration 1787 / 2000) loss: 20.287258
(Iteration 1788 / 2000) loss: 24.330864
(Iteration 1789 / 2000) loss: 18.549137
(Iteration 1790 / 2000) loss: 23.365220
(Iteration 1791 / 2000) loss: 27.172606
(Iteration 1792 / 2000) loss: 24.213697
(Iteration 1793 / 2000) loss: 20.976817
(Iteration 1794 / 2000) loss: 23.174886
(Iteration 1795 / 2000) loss: 24.229418
(Iteration 1796 / 2000) loss: 21.332689
(Iteration 1797 / 2000) loss: 22.656910
(Iteration 1798 / 2000) loss: 22.821440
(Iteration 1799 / 2000) loss: 21.375429
(Iteration 1800 / 2000) loss: 22.252887
(Iteration 1801 / 2000) loss: 24.492862
(Iteration 1802 / 2000) loss: 20.353778
(Iteration 1803 / 2000) loss: 21.080238
(Iteration 1804 / 2000) loss: 21.993685
(Iteration 1805 / 2000) loss: 26.111050
(Iteration 1806 / 2000) loss: 19.917460
(Iteration 1807 / 2000) loss: 22.249453
(Iteration 1808 / 2000) loss: 23.945196
(Iteration 1809 / 2000) loss: 21.350504
(Iteration 1810 / 2000) loss: 21.012098
(Iteration 1811 / 2000) loss: 23.713743
(Iteration 1812 / 2000) loss: 23.140541
(Iteration 1813 / 2000) loss: 25.948671
(Iteration 1814 / 2000) loss: 20.656964
(Iteration 1815 / 2000) loss: 25.784586
(Iteration 1816 / 2000) loss: 24.208963
(Iteration 1817 / 2000) loss: 20.827764
(Iteration 1818 / 2000) loss: 21.002521
(Iteration 1819 / 2000) loss: 21.776450
(Iteration 1820 / 2000) loss: 22.601484
(Iteration 1821 / 2000) loss: 21.459424
(Iteration 1822 / 2000) loss: 21.816599
(Iteration 1823 / 2000) loss: 20.476699
(Iteration 1824 / 2000) loss: 21.359880
(Iteration 1825 / 2000) loss: 21.735897
(Iteration 1826 / 2000) loss: 20.987396
(Iteration 1827 / 2000) loss: 23.715726
(Iteration 1828 / 2000) loss: 22.901460
(Iteration 1829 / 2000) loss: 22.393109
(Iteration 1830 / 2000) loss: 19.910957
(Iteration 1831 / 2000) loss: 21.648183
(Iteration 1832 / 2000) loss: 26.132497
(Iteration 1833 / 2000) loss: 21.751709
(Iteration 1834 / 2000) loss: 22.922526
(Iteration 1835 / 2000) loss: 22.811974
(Iteration 1836 / 2000) loss: 22.758044
(Iteration 1837 / 2000) loss: 26.347222
(Iteration 1838 / 2000) loss: 20.667705
(Iteration 1839 / 2000) loss: 23.160042
(Iteration 1840 / 2000) loss: 23.924654
(Iteration 1841 / 2000) loss: 24.428598
(Iteration 1842 / 2000) loss: 22.271604
(Iteration 1843 / 2000) loss: 24.155433
(Iteration 1844 / 2000) loss: 20.603928
(Iteration 1845 / 2000) loss: 21.628724
(Iteration 1846 / 2000) loss: 21.862092
(Iteration 1847 / 2000) loss: 20.420912
(Iteration 1848 / 2000) loss: 26.657258
(Iteration 1849 / 2000) loss: 18.588650
(Iteration 1850 / 2000) loss: 22.174701
(Iteration 1851 / 2000) loss: 20.567456
(Iteration 1852 / 2000) loss: 24.454751
(Iteration 1853 / 2000) loss: 24.522685
(Iteration 1854 / 2000) loss: 23.204605
(Iteration 1855 / 2000) loss: 21.330746
(Iteration 1856 / 2000) loss: 22.158042
(Iteration 1857 / 2000) loss: 24.293045
(Iteration 1858 / 2000) loss: 24.019324
(Iteration 1859 / 2000) loss: 23.209988
(Iteration 1860 / 2000) loss: 23.712290
(Iteration 1861 / 2000) loss: 26.464497
(Iteration 1862 / 2000) loss: 21.841593
(Iteration 1863 / 2000) loss: 23.272574
(Iteration 1864 / 2000) loss: 20.575277
(Iteration 1865 / 2000) loss: 24.099019
(Iteration 1866 / 2000) loss: 21.886768
(Iteration 1867 / 2000) loss: 22.187933
(Iteration 1868 / 2000) loss: 23.595921
(Iteration 1869 / 2000) loss: 18.806543
(Iteration 1870 / 2000) loss: 20.882701
(Iteration 1871 / 2000) loss: 25.700650
(Iteration 1872 / 2000) loss: 20.824066
(Iteration 1873 / 2000) loss: 22.971407
(Iteration 1874 / 2000) loss: 20.948187
(Iteration 1875 / 2000) loss: 24.440113
(Iteration 1876 / 2000) loss: 23.007748
(Iteration 1877 / 2000) loss: 19.730336
(Iteration 1878 / 2000) loss: 19.720671
(Iteration 1879 / 2000) loss: 24.071330
(Iteration 1880 / 2000) loss: 20.410273
(Iteration 1881 / 2000) loss: 21.509060
(Iteration 1882 / 2000) loss: 20.192226
(Iteration 1883 / 2000) loss: 25.637140
(Iteration 1884 / 2000) loss: 22.450923
(Iteration 1885 / 2000) loss: 22.449787
(Iteration 1886 / 2000) loss: 24.282342
(Iteration 1887 / 2000) loss: 22.433419
(Iteration 1888 / 2000) loss: 23.653429
(Iteration 1889 / 2000) loss: 21.474383
(Iteration 1890 / 2000) loss: 22.123236
(Iteration 1891 / 2000) loss: 21.417370
(Iteration 1892 / 2000) loss: 21.745186
(Iteration 1893 / 2000) loss: 24.248743
(Iteration 1894 / 2000) loss: 19.792954
(Iteration 1895 / 2000) loss: 22.286701
(Iteration 1896 / 2000) loss: 21.725678
(Iteration 1897 / 2000) loss: 23.655255
(Iteration 1898 / 2000) loss: 22.169493
(Iteration 1899 / 2000) loss: 25.962441
(Iteration 1900 / 2000) loss: 23.336021
(Iteration 1901 / 2000) loss: 22.902398
(Iteration 1902 / 2000) loss: 20.653852
(Iteration 1903 / 2000) loss: 22.420030
(Iteration 1904 / 2000) loss: 22.285375
(Iteration 1905 / 2000) loss: 22.996539
(Iteration 1906 / 2000) loss: 23.163317
(Iteration 1907 / 2000) loss: 21.371409
(Iteration 1908 / 2000) loss: 22.007690
(Iteration 1909 / 2000) loss: 23.891420
(Iteration 1910 / 2000) loss: 19.383547
(Iteration 1911 / 2000) loss: 22.584515
(Iteration 1912 / 2000) loss: 20.589272
(Iteration 1913 / 2000) loss: 21.617236
(Iteration 1914 / 2000) loss: 25.372232
(Iteration 1915 / 2000) loss: 21.983091
(Iteration 1916 / 2000) loss: 21.983509
(Iteration 1917 / 2000) loss: 21.617494
(Iteration 1918 / 2000) loss: 19.271644
(Iteration 1919 / 2000) loss: 20.832603
(Iteration 1920 / 2000) loss: 21.388750
(Iteration 1921 / 2000) loss: 19.943772
(Iteration 1922 / 2000) loss: 22.845029
(Iteration 1923 / 2000) loss: 23.717854
(Iteration 1924 / 2000) loss: 23.866286
(Iteration 1925 / 2000) loss: 23.956710
(Iteration 1926 / 2000) loss: 22.504388
(Iteration 1927 / 2000) loss: 21.205293
(Iteration 1928 / 2000) loss: 26.581412
(Iteration 1929 / 2000) loss: 23.554947
(Iteration 1930 / 2000) loss: 22.300802
(Iteration 1931 / 2000) loss: 21.185575
(Iteration 1932 / 2000) loss: 22.483660
(Iteration 1933 / 2000) loss: 18.746267
(Iteration 1934 / 2000) loss: 21.449394
(Iteration 1935 / 2000) loss: 22.468610
(Iteration 1936 / 2000) loss: 17.293693
(Iteration 1937 / 2000) loss: 20.248454
(Iteration 1938 / 2000) loss: 25.701139
(Iteration 1939 / 2000) loss: 27.142200
(Iteration 1940 / 2000) loss: 21.369814
(Iteration 1941 / 2000) loss: 23.484042
(Iteration 1942 / 2000) loss: 22.041265
(Iteration 1943 / 2000) loss: 22.790476
(Iteration 1944 / 2000) loss: 20.570405
(Iteration 1945 / 2000) loss: 19.922643
(Iteration 1946 / 2000) loss: 21.049446
(Iteration 1947 / 2000) loss: 22.670563
(Iteration 1948 / 2000) loss: 24.847017
(Iteration 1949 / 2000) loss: 19.669031
(Iteration 1950 / 2000) loss: 19.246079
(Iteration 1951 / 2000) loss: 24.133362
(Iteration 1952 / 2000) loss: 20.541196
(Iteration 1953 / 2000) loss: 21.666317
(Iteration 1954 / 2000) loss: 22.393155
(Iteration 1955 / 2000) loss: 23.006982
(Iteration 1956 / 2000) loss: 22.393648
(Iteration 1957 / 2000) loss: 20.033631
(Iteration 1958 / 2000) loss: 24.070814
(Iteration 1959 / 2000) loss: 24.765586
(Iteration 1960 / 2000) loss: 20.476157
(Iteration 1961 / 2000) loss: 19.799018
(Iteration 1962 / 2000) loss: 21.408605
(Iteration 1963 / 2000) loss: 24.132998
(Iteration 1964 / 2000) loss: 23.934266
(Iteration 1965 / 2000) loss: 19.904000
(Iteration 1966 / 2000) loss: 24.911767
(Iteration 1967 / 2000) loss: 21.143206
(Iteration 1968 / 2000) loss: 24.537587
(Iteration 1969 / 2000) loss: 21.137135
(Iteration 1970 / 2000) loss: 21.193389
(Iteration 1971 / 2000) loss: 21.879208
(Iteration 1972 / 2000) loss: 21.361886
(Iteration 1973 / 2000) loss: 22.513774
(Iteration 1974 / 2000) loss: 25.752584
(Iteration 1975 / 2000) loss: 16.780888
(Iteration 1976 / 2000) loss: 24.315766
(Iteration 1977 / 2000) loss: 20.698268
(Iteration 1978 / 2000) loss: 20.178218
(Iteration 1979 / 2000) loss: 22.556131
(Iteration 1980 / 2000) loss: 21.890330
(Iteration 1981 / 2000) loss: 21.207501
(Iteration 1982 / 2000) loss: 23.418228
(Iteration 1983 / 2000) loss: 23.446362
(Iteration 1984 / 2000) loss: 25.802801
(Iteration 1985 / 2000) loss: 22.929888
(Iteration 1986 / 2000) loss: 22.124376
(Iteration 1987 / 2000) loss: 25.088521
(Iteration 1988 / 2000) loss: 24.483570
(Iteration 1989 / 2000) loss: 21.627366
(Iteration 1990 / 2000) loss: 20.719790
(Iteration 1991 / 2000) loss: 22.094718
(Iteration 1992 / 2000) loss: 20.852158
(Iteration 1993 / 2000) loss: 24.672810
(Iteration 1994 / 2000) loss: 19.382316
(Iteration 1995 / 2000) loss: 20.810830
(Iteration 1996 / 2000) loss: 22.180600
(Iteration 1997 / 2000) loss: 23.004994
(Iteration 1998 / 2000) loss: 20.243731
(Iteration 1999 / 2000) loss: 21.166581
(Iteration 2000 / 2000) loss: 19.625176
In [13]:
for split in ['train', 'val']:
  minibatch = sample_coco_minibatch(lstm_data, split=split, batch_size=2)
  gt_captions, features, urls = minibatch
  gt_captions = decode_captions(gt_captions, data['idx_to_word'])

  sample_captions = lstm_model.sample(features)
  sample_captions = decode_captions(sample_captions, data['idx_to_word'])

  for gt_caption, sample_caption, url in zip(gt_captions, sample_captions, urls):
    plt.imshow(image_from_url(url))
    plt.title('%s\n%s\nGT:%s' % (split, sample_caption, gt_caption))
    plt.axis('off')
    plt.show()